Actually, most elements can be affected by magnetic fields (paramagnetism)

Apr 9, 2020
12:15:57am

ThetaSigma All-American

Essentially, if a certain electron orbit is not completely full (which is the case in most atoms), then all the unpaired electrons in that orbit have the same spin, and so the sum total of the reactions of electrons within the atom to a magnetic field is not zero, and so an external magnetic field can exert a force on that atom. That's called paramagnetism, and it's not what we usually mean when we talk about magnetism and magnets.

Wikipedia

Paramagnetism

Paramagnetism is a form of magnetism whereby some materials are weakly attracted by an externally applied magnetic field, and form internal, induced magnetic fields in the direction of the applied magnetic field. In contrast with this behavior, diamagnetic materials are repelled by magnetic fields and form induced magnetic fields in the direction opposite to that of the applied magnetic field. Paramagnetic materials include most chemical elements and some compounds; they have a relative magnetic permeability slightly greater than 1 (i.e., a small positive magnetic susceptibility) and hence are attracted to magnetic fields. The magnetic moment induced by the applied field is linear in the field strength and rather weak. It typically requires a sensitive analytical balance to detect the effect and modern measurements on paramagnetic materials are often conducted with a SQUID magnetometer.

https://en.wikipedia.org/wiki/Paramagnetism

Usually, when we talk about magnetism, we're talking about ferromagnitism, which is the type of magnetism that permanent magnets are involved in. Everyday ferromagnetic materials are all metal, so you're right in that sense. (But there are materials other than metal that can be ferromagnetic--see the wikipedia article.)

Wikipedia

Ferromagnetism

Ferromagnetism is the basic mechanism by which certain materials (such as iron) form permanent magnets, or are attracted to magnets. In physics, several different types of magnetism are distinguished. Ferromagnetism (along with the similar effect ferrimagnetism) is the strongest type and is responsible for the common phenomenon of magnetism in magnets encountered in everyday life. Substances respond weakly to magnetic fields with three other types of magnetism—paramagnetism, diamagnetism, and antiferromagnetism—but the forces are usually so weak that they can be detected only by sensitive instruments in a laboratory. An everyday example of ferromagnetism is a refrigerator magnet used to hold notes on a refrigerator door. The attraction between a magnet and ferromagnetic material is "the quality of magnetism first apparent to the ancient world, and to us today".Permanent magnets (materials that can be magnetized by an external magnetic field and remain magnetized after the external field is removed) are either ferromagnetic or ferrimagnetic, as are the materials that are noticeably attracted to them. Only a few substances are ferromagnetic. The common ones are iron...

https://en.wikipedia.org/wiki/Ferromagnetism

There are also other types of magnetism, but I don't understand the others so you'll have to ask someone else or read the wikipedia articles on them.