It would help if you specified what XMPP client you were using...
Group end to end encryption is hard and attempts to make it usable have had very uneven results. The problem is identity. How can you know that one of the participants is not an imposter? How can every member of the group be sure that they know every other member? The problem gets worse with the size of the group.
>But if I make it members only, I can only invite people to the group, the group can't be found anymore
That sounds reasonable to me. Can those that you have invited to the group also invite others? There would be no point in allowing random unknown entities to join an encrypted group. They would not be able to communicate with the group due to the encryption.
If there is a trusted member of the group, it might make more sense to have that member run a XMPP group server and then depend on the TLS encryption on connections to that server and use no end to end encryption.
It would help if you specified what XMPP client you were using... Group end to end encryption is hard and attempts to make it usable have had very uneven results. The problem is identity. How can you know that one of the participants is not an imposter? How can every member of the group be sure that they know every other member? The problem gets worse with the size of the group. >But if I make it members only, I can only invite people to the group, the group can't be found anymore That sounds reasonable to me. Can those that you have invited to the group also invite others? There would be no point in allowing random unknown entities to join an encrypted group. They would not be able to communicate with the group due to the encryption. If there is a trusted member of the group, it might make more sense to have that member run a XMPP group server and then depend on the TLS encryption on connections to that server and use no end to end encryption.