Honestly, Rare's handheld efforts were never that strong to begin with.
It should be noted that their handheld games were handled by a D-team within Rare seperate from any of their main developers. This chart provides an interesting insight to Rare's work culture back in the N64 era to the early 2000s.
But yeah, I never really thought Rare's handheld stuff was ever that great, despite a being a fan of their console output. A lot of them just felt like watered down versions of better games like DKC, Perfect Dark or Banjo. Not to mention how kind of butchered the DKC GBA ports and especially the Diddy Kong Racing DS remake were in comparison to their original versions.
I don't think even Rare themselves really cared about them. Banjo-Kazooie: Nuts & Bolts features tons of references and callbacks to Banjo-Kazooie and Banjo-Tooie and yet there's no acknowledgement of Grunty's Revenge, even though that game was written as an interquel between the N64 Banjo games. Even Grunty laments at some point that "a three time loser [she] cannot be", which kind of implies GR isn't canon lol.