Seems like I need to start D-boxing some rare valuables before opening these.
I'm pretty sure you won't lose the item even if its a duplicate.
If your inventory is full or you already have the item the NPC will just hold the item until can retrieve it.
I don't think the system would give you a duplicate item, so long as its in your inventory and is Rare. Reason being, say you have Crimson Cruisses. You trade your SP key to the goblin, and he attempts to give you Crimson Cruisses. Since the item is Rare, you cannot obtain another, and the goblin tells you to return after sorting your inventory. But, he doesn't tell you what item you were supposed to get, so you don't know what to send to a mule or toss. Since you don't know this, you either have to send a few items at a time until you send the correct item, or the goblin is locked and you can't use him. Any time you attempt to talk to him in the future, before any menu options come up, he attempts to give you the item he is holding, and will not proceed beyond this stage until the item is delivered.