[python] aws endpoint dns record automation module

Seunghyun Moon·2022년 10월 12일
0

python

목록 보기
2/4

an interesting mission has been delivered to me.
one of the clients who had decied to use our solution uses FWs that only supports ip based rules.

in normal cases one only needs one or few public ips to use a solution. but this case our solution needs api access to CSPs.(aws, azure, ncp etc)
this is where it gets complicated. they need actual ips!(not domain) and the solution communicates with more than 100 api endpoints...

here comes the best part. naturally I, the best one to solve the problem, was summoned.
here's the mission statement.

we'll get you a list on domains that we use. you nslookup, get the ips, run tests and documentate them.(showing me the following screen.

no time to sigh. get coffee and let's get started.

yes I didn't expect it to be easy. I see irregularities. every time I look ip up I get different results and there are so many cases. also I like to automate.

I've decided to write some codes.
the language will be Python, easy and supports many handy libraries.

Libraries

import pandas as pd
import subprocess
import re
import ipaddress

pandas

maps ips to domains also I needed dataframe.

subprocess

to run nslookup command like it does the server.

re

to preprocess dns text record values.

ipaddress

to validate if it's a valid ip address.

DNS query

tried some other ways and luckily aws provides text records that seem to give a result of all ips an endpoint is pointing.

lookup and validate

puts nslookup results into res. then preprocess them.
finally validates if it's a valid ip address.

used two try except clauses to see which error retrieves each loop.

for url in test:

    try:
        res=subprocess.getstatusoutput("nslookup -query=txt "+url)[1]

        res=res[res.find("text = ")+7:].replace('"','').replace(' n1\n\nAuthoritative answers can be found from:\n','')
        res=re.sub(r'c[0-9]{,3}', '', res)
        res=re.sub(r'e[0-9]{,1}', '', res)

        for ip in res.split(' '):
            try:
                ipaddress.IPv4Network(ip)
                testresult.append((url, ip))
            except:
                print("first except: ",url)
                print("and the ip looks like this: ",ip)

    except:
        print("second except: ",url)

save to .csv format

converts mapped ip-domain into dataframe, save it to csv file.

print("done looping")

df = pd.DataFrame(testresult,columns=['url','ip'])
df.to_csv("somefilename.csv", mode='w')
print("done converting")

exception handling

not every domain had a text record. thus I give the computer the job what it was made for.

awsiam=['iam.amazonaws.com','iam-fips.amazonaws.com','iam.us-gov.amazonaws.com']

for i in range(2000):
    if i%10==0:
        print(i,'th loop done')

    for url in test:
        try:
            testresult.append((url,socket.gethostbyname(url)))
        except:
            print(url)

takeaway

I guess it's not the smartest way. although it works. It had passed all test procedures.(making aws sg(security groups) that has 800 ips was another challenge. we'll see this on another post.)

profile
I live fullest

0개의 댓글