The price is coming from the hardware, which I think most of us agree is fair for the quality of the materials. The main issue (leaving the case design aside) is the lack of games to back that up, as good as AvP and Progear are. Simply adding more games they already own all the rights to wouldn't drastically increase the development time or complexity enough to cause a significant dent in how much they'd charge, because in the end, it's just FBAlpha, a single program that'll run any CPS2 ROM they throw at it. At most, they'd have to add the entries to whatever game list it has and QA would have more work to do, which is actually probably the most expensive part.
Well, I just found one through Google in about a minute that advertised itself as having 800 games for about £230 (obviously I won't link it here), which is around the same price. I can find them on Amazon for anywhere from £110 to £200 with varying sets of games. I don't own or want one, they're trash, but a large part of the potential audience for a device like this aren't so discerning.
