Probably yes, it depends on your threat model.

If you are using E2EE on a matrix.org account then your message content, attachments (images) and most other traffic isn't accessible to anyone but the people in the chat. However Matrix isn't the most private option, it has a number of leaks such as reactions and chat topics (these are being worked on but aren't close to happening).

For most people Matrix is a very private and secure option and the fact that it is federated is a huge plus. If you want something more secure you are probably looking at Signal (which you don't want to use and isn't federated) or Simplex Chat (which doesn't have multi-device support).

you don't need to use matrix.org. there are several open homeservers, like chat.mozilla.org, but also there are people who host services for others to use. you may have a look at current lemmy hosts, and their other services if they have them.

Why would Matrix be the only option? XMPP is significantly better. You can either sign up on a public server or pay a small sum to have your own private server for you and your family for example on https://snikket.org/ or I think https://jmp.chat/ also includes optionally a small server in the subscription.

Yeah, sure. But Matrix is decentralized and federated. So you can pretty much join any instance and be able to talk with anyone on any instance. So why not select another instance ~~or maybe even self host one yourself?~~

edit: didn't read the text till the end

Matrix is great, you can use another instance though.

https://servers.joinmatrix.org/