Mammoth Uncensored Word List:
8 words ending with ya, 5 letters long

abaya aliya bunya chaya khaya playa shaya thuya