Doubtful as the law would likely need updated to allow for that.
They'd basically have no way to enforce the punitive counter-measures. They could certainly hit a company with an invoice, but that company could just ignore it. As Github ignoring future 1201 violation claims from that company would open Github up to liability, regardless of the reason for ignoring the claims.
Just spitballing, but maybe Tortuous Interference. The Youtube-DL developers and users have an advantageous business relationship with GitHub/Microsoft, which was interrupted and permanently harmed by the frivolous claim. Github has done this to mitigate the harm, but people will continue to question whether Microsoft will reliably serve, etc.
YouTube-DL could potentially argue Slander of Title, which is well established in terms of claiming ownership of another person's copyright. Claiming someone else's intellectual property is inherently illegal seems pretty similar.
It would be hard for YouTube-DL to prove damages, but with a showing of intent there could be room for punitive damages based on what the RIAA thought they stood to gain.
They could throw few millions at lawyers to draw a law that makes organisations like RIAA illegal and then put forward few more millions at congressmen and lobby for its passage. That would make real difference. RIAA estate should be confiscated, sold and proceeds distributed among artists (not labels)
Their new claim processing appears to be fairly resource-heavy. It'd be only fair to recover some of the costs they will end up sinking into it.