Sort a list of domain names (FQDNs) starting from tld and working left

Question

I am looking to sort a list of domain names (a web filter whitelist) starting from the TLD and working upwards. I am looking any *nix or windows tools that can do this easily, though a script would be fine too.

So if the is the list you are given

www.activityvillage.co.uk 
ajax.googleapis.com 
akhet.co.uk 
alchemy.l8r.pl 
au.af.mil 
bbc.co.uk 
bensguide.gpo.gov 
chrome.angrybirds.com 
cms.hss.gov 
crl.godaddy.com 
digitalhistory.uh.edu 
digital.library.okstate.edu 
digital.olivesoftware.com

This is what I want as the output.

chrome.angrybirds.com 
crl.godaddy.com 
ajax.googleapis.com 
digital.olivesoftware.com 
digital.library.okstate.edu 
digitalhistory.uh.edu 
bensguide.gpo.gov 
cms.hss.gov 
au.af.mil 
alchemy.l8r.pl 
www.activityvillage.co.uk 
akhet.co.uk 
bbc.co.uk

Just in case you are wondering why, Squidguard, has a bug/design flaw. If both www.example.com and example.com are both included in a list, then the example.com entry is ignored and you can only visit content from www.example.com. I have several large lists that need some cleanup because someone added entries without looking first.

Answer 1

This simple python script will do what you want. In this example I name the file domain-sort.py:

#!/usr/bin/env python
from fileinput import input
for y in sorted([x.strip().split('.')[::-1] for x in input()]): print('.'.join(y[::-1]))

To run it use:

cat file.txt | ./domain-sort.py

Note that this looks a little uglier since I wrote this as more or a less a simple one-liner I had to use slice notation of [::-1] where negative values work to make a copy of the same list in reverse order instead of using the more declarative reverse() which does it in-place in a way that breaks the composability.

And here's a slightly longer, but maybe more readable version that uses reversed() which returns an iterator, hence the need to also wrap it in list() to consume the iterator and produce a list:

#!/usr/bin/env python
from fileinput import input
for y in sorted([list(reversed(x.strip().split('.'))) for x in input()]): print('.'.join(list(reversed(y))))

On a file with 1,500 randomly sorted lines it takes ~0.02 seconds:

Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.02
Maximum resident set size (kbytes): 21632

On a file with 150,000 randomly sorted lines it takes a little over 3 seconds:

Elapsed (wall clock) time (h:mm:ss or m:ss): 0:03.20
Maximum resident set size (kbytes): 180128

Here is an arguably more readable version that does the reverse() and sort() in-place, but it runs in the same amount of time, and actually takes slightly more memory.

#!/usr/bin/env python
from fileinput import input
data = []
for x in input():
   d = x.strip().split('.')
   d.reverse()
   data.append(d)
data.sort()
for y in data:
   y.reverse()
   print('.'.join(y))

On a file with 1,500 randomly sorted lines it takes ~0.02 seconds:

Elapsed (wall clock) time (h:mm:ss or m:ss): 0:00.02
Maximum resident set size (kbytes): 22096

On a file with 150,000 randomly sorted lines it takes a little over 3 seconds:

Elapsed (wall clock) time (h:mm:ss or m:ss): 0:03.08
Maximum resident set size (kbytes): 219152

Answer 2

Here's a PowerShell script that should do what you want. Basically it throws all the TLD's into an array reverses each TLD, sorts it, reverses it back to its original order, and then saves it to another file.

$TLDs = Get-Content .\TLDsToSort-In.txt
$TLDStrings = @();

foreach ($TLD in $TLDs){
    $split = $TLD.split(".")
    [array]::Reverse($split)
    $TLDStrings += ,$split
}

$TLDStrings = $TLDStrings|Sort-Object

foreach ($TLD in $TLDStrings){[array]::Reverse($TLD)}

$TLDStrings | %{[string]::join('.', $_)} | Out-File .\TLDsToSort-Out.txt

Ran it on 1,500 records - took 5 seconds on a reasonably powerful desktop.

Answer 3

cat domain.txt | rev | sort | rev

Answer 4

Slightly less cryptic, or at least prettier, Perl:

use warnings;
use strict;

my @lines = <>;
chomp @lines;

@lines =
    map { join ".", reverse split /\./ }
    sort
    map { join ".", reverse split /\./ }
    @lines;

print "$_\n" for @lines;

This is a simple example of a Guttman–Rosler transform: we convert the lines into the appropriate sortable form (here, split the domain name on periods and reverse the order of the parts), sort them using the native lexicographic sort and then convert the lines back to their original form.

Answer 5

In Unix scripting: reverse, sort and reverse:

awk -F "." '{for(i=NF; i > 1; i--) printf "%s.", $i; print $1}' file |
  sort |
  awk -F "." '{for(i=NF; i > 1; i--) printf "%s.", $i; print $1}'

Answer 6

Here it is in (short and cryptic) perl:

#!/usr/bin/perl -w
@d = <>; chomp @d;
for (@d) { $rd{$_} = [ reverse split /\./ ] }
for $d (sort { for $i (0..$#{$rd{$a}}) {
        $i > $#{$rd{$b}} and return 1;
        $rd{$a}[$i] cmp $rd{$b}[$i] or next;
        return $rd{$a}[$i] cmp $rd{$b}[$i];
} } @d) { print "$d\n" }

Answer 7

OK, I'll add one more solution (after 10 years), because the other python solutions did not mention sorted key-function which is readable and easy to understand:

#!/usr/bin/env python
import sys
def kf(v):
    """Key-function to sort domains in tld order"""
    return '.'.join(reversed(v.strip().split('.')))
sys.stdout.writelines(sorted(sys.stdin, key=kf))

Answer 8

awk -F"." 's="";{for(i=NF;i>0;i--) {if (i<NF) s=s "." $i; else s=$i}; print s}' <<<filename>>> | sort | awk -F"." 's="";{for(i=NF;i>0;i--) {if (i<NF) s=s "." $i; else s=$i}; print s}'

What this does is to reverse each filed in the domain name, sort and reverse back.

This truly sorts the domain list, lexicographically based on each part of the domain-name, from right to left.

The reverse solution (rev <<<filename>>> | sort | rev) , does not, I've tried it.

Answer 9

One more answer, using awk's built-in sorting to sort each part of the name lexically:

awk -F. '{k="";for(x=NF;x>0;x--){k=k"."$x};o[k]=$0};END{PROCINFO["sorted_in"]="@ind_str_asc";for(n in o){print o[n]}}' < infile > outfile

More readably:

awk -F. '
  {
    k=""
    for(x=NF;x>0;x--){k=k"."$x}
    o[k]=$0
  }
END{
    PROCINFO["sorted_in"]="@ind_str_asc"
    for(n in o){
      print o[n]
    }
  }
' < infile > outfile

On my system, awk is really GNU Awk 5.1.0; and it's unclear what versions of awk/mawk/nawk/gawk/etc support PROCINFO["sorted_in"]. However, the first awk I ever used would sort the output alphabetically by the array key, so PROCINFO is really only there to prevent 'modern' versions of gawk from messing up the output. :-)

Answer 10

In JavaScript:

function compareName(a, b) {
    const aReverse = a.split('.').reverse().join('.')
    const bReverse = b.split('.').reverse().join('.')
    return aReverse.toLowerCase().localeCompare(bReverse.toLowerCase())
}
const myNames = ['a.example', 'Z.a.example', 'yl.a.example', 'example']
myNames.sort(compareName)

Uses the same algorithm as @aculich's answer.