As self.g = n + 1, would it be better to replace
return invert(self.l_function(powmod(self.public_key.g, x - 1, xsquare),x), x)
with
return invert(self.l_function((self.n * (x - 1) + 1) % xsquare , x), x)
in phe/paillier.py/h_function(self, x, xsquare), it may allow it to be faster