I'm no expert for sure. For years I thought the fretwork was wood.

It's actually plastic on mine and the ones I've seen up close. There are finishing nails in mine and since replacement grills are unobtanium I've decided to leave mine alone for now.
The plastic becomes quite brittle with age. Mine are over 40 years old so I'll think long and hard before I mess with them. I have another pair with no fretwork so I need to refinish the cabinets and make new grills of some sort.
If I decide to remove the fretwork I'll either use a thin pin punch or a real small nail set to drive the brads on through the fretwork. If you decide to go ahead remember to take your time. You only need to miss once to have a problem.
If you plan on re-installing the fretwork then I'd be sure and meaure the brads you use and drill a hole through the fretwork so the nail doesn't split it. The drill bit needs to be real close to the O.D. of the brads but not large enough to allow the small head to pass through it.
Mike