There's only one point & click adventure game that I know on Windows 3.1, and it's Uncle Julius and the Anywhere Machine. Granted it's quite different from what you describe but maybe it can help you track your game down.
Also, if not already, I recommend you to visit this website:
https://www.reddit.com/r/tipofmyjoystick/
People can help you track down your game in r/tipofmyjoystick.
The Anywhere Machine:
https://www.youtube.com/watch?v=Xs4ad3zibts
It's downloadable free of charge:
https://familygames.com/classic
__________________
Reverend Preacherbot: Wretched sinner unit! The path to Robot Heaven lies here, in the Good Book 3.0.
Bender: Hey. Do I preach at you when you're lying stoned in the gutter? No!