Threatintel

Background Many years ago, I learned about MISP Warning Lists, which is an awesome and actively maintained project. It aggregates many public datasets of well-known indicators, including: Network subnets of popular Cloud providers Top websites lists Networks of many VPN providers Reserved IP subnets And many more. Shout out to the CIRCL.lu folks, who are doing a really great job! The Challenge Originally, these lists are intended as a supplement to the MISP framework, but I always wanted to use them as a standalone tool....

Sometimes you need to gather threat intelligence data as quickly as possible, and Rapid7’s Project Sonar Opendata can provide great insights. However, there’s a challenge: you can’t easily grep the HTTP response body with the lovely jq tool because the data field in the resulting JSON is base64 encoded: { "data": "SFRUUC8xLjAgNDAwIEJhZCBSZXF1ZXN0DQpTZXJ2ZXI6IEFrYW1haUdIb3N0DQpNaW1lLVZlcnNpb246IDEuMA0KQ29udGVudC1UeXBlOiB0ZXh0L2h0bWwNCkNvbnRlbnQtTGVuZ3RoOiAyMDgNCkV4cGlyZXM6IE1vbiwgMjMgQXByIDIwMTggMDc6NDA6MjkgR01UDQpEYXRlOiBNb24sIDIzIEFwciAyMDE4IDA3OjQwOjI5IEdNVA0KQ29ubmVjdGlvbjogY2xvc2UNCg0KPEhUTUw+PEhFQUQ+CjxUSVRMRT5JbnZhbGlkIFVSTDwvVElUTEU+CjwvSEVBRD48Qk9EWT4KPEgxPkludmFsaWQgVVJMPC9IMT4KVGhlIHJlcXVlc3RlZCBVUkwgIiYjOTE7bm8mIzMyO1VSTCYjOTM7IiwgaXMgaW52YWxpZC48cD4KUmVmZXJlbmNlJiMzMjsmIzM1OzkmIzQ2OzFmMzEzMjE3JiM0NjsxNTI0NDY5MjI5JiM0NjsyNDFiOGFmCjwvQk9EWT48L0hUTUw+Cg==", "host": "REDACTED", "ip": "REDACTED", "path": "/", "port": 80, "vhost": "REDACTED" } While you could probably grep this using a decent bash script, I believe I have a better option....