I just tried it again, for realsies this time. The strategy wasn't working on insane, where the kobolds were one shotting me (disconcerting!), however on hard what you need to do is tag (damage or run into) the goblins before they get to the crystal. On hard, I was able to just stand in the north circle/doorway and all the kobolds would scrape up against me and agro me, dealing 2 damage (98% block). For the southern kobolds, you would need to stand a bit ahead of the stairs so they have a chance to agro you before the crystal, along with something like the pyro pet which should be able to agro vast sums of kobolds.