flask, CSV parsing, think like a programmer

by Joost Agterhoek — on  ,  ,  , 

cover-image

It's been a while! I expect that to be a regular opening line here, unfortunately. But it doesn't really matter, as I've kept busy and made some significant progress lately. While there is nothing definitive (or 'release worthy'), I would like to quickly jot down what I've learned and have been working on the last months.

Basically: I try to think more like a programmer. Meaning, why would I do something by hand in five minutes when I can spend an hour and a half to automate it? Honestly, it's not easy, but I feel this is a great way to work programming into my everyday work and make the same mistakes over and over until I don't anymore. What does that mean? Well:

Python

flask website to look up (malicious) IP addressess with URI_lookup and (soon) vt-py

PySimpleGUI to look up (malicious) IP addressess with IPWhois and (soon) vt-py

extracting, parsing and listing (sub)domains, email addressess and other data from text, csv and xlsx-files with pandas and list comprehension

checking SPF and DMARC records of URLs with checkdmarc

grep/awk/sed

grep data from html tables and filter unique identifiers (like email addressess) with awk

learning awk and sed syntax to extract email addressess and filter on last names:

awk 'BEGIN {FS=">|<"}/@domain.tld/{if ($3!="my.name@domain.tld") print $3}' *.html | sed 's/.*\.\([^.]*\)@/\1 &/' | sort -k 1,1 | cut -d ' ' -f2-

Azure Logic Apps and Word templates

Active Content controls in Word documents for templating with form data from Azure Logic App

PowerShell oneliners equivalent to Linux tools

head = Get-Content 'FileName' | Select-Object -First 'NumberOfLines'

tail = Get-Content 'FileName' | Select-Object -Last 'NumberOfLines

Some useful Windows Terminal shortcuts from File Explorer

Windows Terminal in your current folder: Ctrl + L -> type 'wt -d .'

Windows Terminal with profile of choice in current folder: Ctrl + L -> type 'wt --profile "profileName" -d .