If the supply network were the problem, all the voltages would change in proportion and in sync, and your lights and appliances would be showing symptoms. I still think the problem is bad connections so I would be disconnecting the power and testing continuity / resistance of all the conductors.
One work-around you might be able to use for the time being, that would get you two speeds, is as follows:
Disconnect and securely insulate the problem L3 wire at the Perilex outlet. Then at the switch, connect one of the blacks to the working permanent brown live feed. Find that black wire at the Perilex and use it as the new L3 permanent feed. Use the other black for whichever of L1 or L2 gives the more useful speed.
Last night I have turn all the fuses down except this one .. and try to understand if something else is causing this issue. No, nothing has changed.
This morning, I have checked the voltage on brown and get 200V instead of 230s in the mid night..
I really dont know how to test all the conductors and resistance as I have no access the inside of the fuse box as well as the perilex outlet and the switch is 20m apart...
I was thinking about using two speed, let say low/ high while discarding the brown cable once for all .. and using the black ones only ... but I just got stuck with that and even can not work, cant get this out of my head ...
May be I should ask for professional help ..
A couple of weeks ago, I had a strange earth leakage circuit breaker issue as well for the first group.. without any reason, the it gone down for a week even we were not at home ... then It came back to normal .. As this is on another group and all the devices working fine, I dont see any relation with that though..
So confused and stuck ...