Making Japanese friends

The longer I live here and the older I get, the harder I feel it has become to make Japanese friends. I have two friends who I met while I was in my own country who I am still in contact with today but I find that all the people that I have met during my time in Japan have moved on or we'll fallen out of touch. I'm not particularly girly and I'm no longer the clubbing and drinking type either. I can speak conversational Japanese but I do feel like I have to be careful with what I say because I find the language restricting in some way. It would be nice to make some local friends who are Japanese but I don't know if I feel hopeful.