Question regarding setting up electrum hardware wallet.

I recently built and am running my own node, and I would like to set up an electrum wallet with my ledger hardware wallet. My bitcoin is currently in my ledger live bitcoin wallet, however I am in the process of creating a hardware wallet with electrum. The install wizard asks if I would like set wallet file encryption but warns that if I do this, I will not be able to access this wallet without the hardware device. Is it recommended to do this? I ask because I want to be able to access my node remotely and also my electrum wallet without having my hardware wallet on me at all times. Therefore should I not select this encryption option? Is it not wise to not select this? Thanks for your help.