Directory Brute Forcing with common tools:

Finding non-linked resources is an important part of any assessment. If you’re working with a scope that limits tools you can install/use, or you want to comb over some could-be false positives from a tools output you can do this by using a bash script. 

Resource lists can be gathered from common tools like wfuzz, dirbuster, and grendelscan. In addition reformatting the [[http://yokoso.inguardians.com/|Yokoso!]] fingerprints can yield tremendous results when on an internal test and pointing this at servers you have found. 

# cat dircurl.sh
<code>
#!/bin/bash

if [[ $# -ne 2 ]]; then
echo "usage: $0 directorylist www.target.com"
exit
fi

for i in $(cat $1)
do echo -ne "directory: "
echo -ne $i
echo -ne "\t"
echo -ne "count: "
echo -ne `curl $2/$i 2> /dev/null | wc -l`
echo
done
</code>

This does a Curl request to each line in the supplied “directorylist” to the “target.com” and then does a wordcount (wc -l) on it. Look at the output, what is the most common response?

<code>
# bash dircurl.sh scanneroutput www.securityaegis.com

directory: sitemap      count: 266
directory: archives     count: 266
directory: wp-admin     count: 7
directory: links        count: 0
directory: login        count: 266
directory: articles     count: 266
directory: support      count: 266
directory: keygen       count: 266
directory: article      count: 266
directory: help count: 266
directory: events       count: 266
directory: archive      count: 266
directory: register     count: 266
directory: en   count: 266
directory: forum        count: 266
directory: wp-includes  count: 7
directory: software     count: 266
directory: downloads    count: 266
directory: security     count: 0
directory: category     count: 266
directory: content      count: 266
directory: main count: 266
directory: press        count: 266
directory: media        count: 266
directory: templates    count: 266
directory: services     count: 266
directory: icons        count: 266
directory: wp-content   count: 7
directory: resources    count: 0
directory: info count: 0
directory: overnment    count: 266
directory: corrections  count: 266
directory: ajax count: 266
directory: icom_includes        count: 266
directory: rules        count: 266
directory: tr   count: 266
directory: server       count: 266
directory: mirrors      count: 266
directory: government    count: 266
directory: corrections  count: 266
</code>

Looks like my error page (or in some cases my redirects) have about 266 newlines. Lets pipe that into grep -v 266, removing all lines containing 266:

<code>
# bash dircurl.sh scanneroutput www.securityaegis.com |grep -v 266

directory: wp-admin     count: 7
directory: links        count: 0
directory: wp-includes  count: 7
directory: security     count: 0
directory: wp-content   count: 7
directory: resources    count: 0
directory: info count: 0
</code>

This gives us a good place to start poking for non-linked resources.