You will not get withdrawals from them, but taking them and then stopping can make falling asleep more difficult. I usually use sedating antihistamines when coming off other drugs
and basically slowly adjust the dose down.
I should note that I have had life long sleeping problems and usually have to take at least something to help me sleep. In fact, I'd say sleep is the biggest factor in contributing to the periods of addiction
I have had in my life.
I do have to say, I am finding I am liking unisom more and more. I find Diphenhydramine
tends to make you sleepier quicker but unisom has a better quality of sleep. The trade off being that unisom will hang with you longer the next day.