How to create ZIP archive with relative paths

One question in SE site give me idea how to zip folder w/o including the current directory in the path. Here is the question:

I have a directory called folder that looks like this:

folder
      -> root_folder
                    -> some files

I want to zip this directory into zipped_dir..

And the answer I found was sample and effective:

Try to use this command (you will get the idea)

cd folder; zip -r ../zipped_dir.zip *

Maybe there is other way, but this is fastest and simplest for me :)

The lesson is that people must learn how to use relative directories :)
P.S. Here is the link to the Q/A in SE .

Beyond the limits - environment (cont 2)

Comment from Paul for the article make me rerun the tests in different conditions: I update the memory of both machines to 1024 MB and use scripts (see below)

[root@centos ~]# cat z1.sh
TEMPVAR=A
for i in {1..1000};
        do
        TEMPVAR=${TEMPVAR}$TEMPVAR;
        a=$(echo 2^$i|bc)
        b=${#TEMPVAR}
        if [ $a -ne $b ]
        then echo KO - $a, $b
        else echo $a
        fi
done
[root@centos ~]# cat z1.1.sh
TEMPVAR=A
for i in {1..1000};
        do
        TEMPVAR=${TEMPVAR}$TEMPVAR;
        a=$(echo 2^$i+1|bc)
        b=$(echo $TEMPVAR|wc -c)
        if [ $a -ne $b ]
        then echo KO - $a, $b
        else echo $a
        fi
done

Solaris
On Solaris independently of the amout of memory and the way of get length of environment variable the result is same:

bash-3.00# vmstat 1 2
 kthr      memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr cd f0 s0 --   in   sy   cs us sy id
 0 0 0 1075080 616428 152 3085 0 0 0  0 204 10 -0 3  0  309 3345 1511 14 19 67
 1 0 0 1407772 753456 18 50  0  0  0  0  0  0  0  0  0  304  324  142  0  1 99

It seems 32MB is internal limitation in bash (or OS) in Solaris

bash-3.00# ./z1.sh
2
4
<snip>
8388608
16777216
33554432
./z1.sh: fork: Not enough space
./z1.sh: line 7: [: -ne: unary operator expected
./z1.sh: fork: Not enough space
bash-3.00# ./z1.1.sh
3
5
<snip>
16777217
33554433
./z1.1.sh: fork: Not enough space
./z1.1.sh: fork: Not enough space
KO - ,
./z1.1.sh: fork: Not enough space

And the execution times follow the common sense

bash-3.00# time ./z1.sh
2
4
<snip>
16777216
33554432
real    0m9.670s
user    0m6.454s
sys     0m3.014s
bash-3.00# time ./z1.1.sh
3
5
<snip>
16777217
33554433
real    0m13.709s
user    0m8.040s
sys     0m5.352s


Linux
But on linux

[root@centos ~]# vmstat 1 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0  10416 1016704    252   4072   75  804   318   828 1008   77 19 44 32  4  0
 0  0  10416 1016704    252   4096    0    0     0     0 1015   14  0  1 99  0  0


my method of count permit me to reach bigger length

[root@centos ~]# ./z1.sh
2
4
<snip>
67108864
134217728
./z1.sh: xmalloc: cannot allocate 536870913 bytes (0 bytes allocated)
[root@centos ~]# ./z1.1.sh
3
5
<snip>
134217729
268435457
./z1.1.sh: xmalloc: cannot allocate 1073741825 bytes (0 bytes allocated)
KO - 536870913, 0
./z1.1.sh: xmalloc: cannot allocate 1073741825 bytes (0 bytes allocated)
 

and need less execution time (with limit to 2^26)

[root@centos ~]# time ./z1.sh
2
4
<snip>
33554432
67108864
real    1m58.730s
user    0m33.084s
sys     1m27.562s
[root@centos ~]# time ./z1.1.sh
3
5
<snip>
33554433
67108865
real    1m36.457s
user    0m50.206s
sys     0m47.815s



Final conclusion: this need further investigations :)

Beyond the limits - environment (cont)

4. Accessable command line argument
Now I will try to see how many are acceptable command line arguments from system utils like pgrep, pkill and similar. For this puspose I will use dummy C program which do nothing

#include <stdio.h>
int main(int argc, char *argv[])
{
       while(1){};
}
 

And will generate sequence of random numbers to be fure will have ho false results

for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|md5sum|awk '{printf $1" "}'>>random_arg;done
[root@centos ~]# cat random_arg
d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09
<snip>
1187d29932b7d5827080684f16205330 92398e4719194f13a1a79064c5a69d05 e1cd425a036f77cc6fb6f3002d00bada a0c6d3b48c5e8146b2fe0b5f74383b9c

Let try with shorter arguments

4.1. Linux
Compile the program
[root@centos ~]# cc z4.c

 And execute it
[root@centos ~]# ./a.out `cat random_arg `&
[1] 5094
 
Lets check what I can see with ps
[[root@centos ~]# ps -efl|grep a.out
0 R root      5094  2174 99  85   0 -   403 -      06:13 pts/0    00:00:35 ./a.out d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09 73e522501878712898d1c47342b9bd66
<snip>
f566402f18f3ebe15453d3a74fc1c534 f87fe9ab1f5f85238ce01ca5d62d57e1 d05d9d3b9cb608d8a177434a4cda5c69 43e80e6069a763a0da17abdba8a06b30 d79a21984d368e4b574e4690a749c7af f5c9eddc01dd1a98bdb7ecbd5eba87a7 249a93f99bb66ff2361f722d1de2b
0 R root      5101  2174  0  78   0 -  1001 -      06:13 pts/0    00:00:00 grep a.out
 
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
    if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
        then echo OK $CNT
        else echo $i
    fi
    CNT=`echo $CNT + 1|bc`
done
 
 and exec
[root@centos ~]# ./z3.sh
OK 1
OK 2
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
 
So I can see only 123 arguments. Hm, this a big number, but no so much
Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
    if [ `pgrep -f $i|wc -l` -eq 1 ]
        then echo OK $CNT
        else echo $i
    fi
    CNT=`echo $CNT + 1|bc`
done

[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
 
The same result. So i can see beyond parameter number 123. And sample chech if this depend of number of variables in environment
[root@centos ~]# env|wc -l
20
[root@centos ~]# export BB=1
[root@centos ~]# env|wc -l
21
[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>

It seems do not depend and this is internal limitation



4.2. Solaris
Compile the program
bash-3.00# /usr/sfw/bin/gcc z3.c
and generate random numbers

for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|digest -a md5|awk '{printf $1" "}'>>random_arg;done

bash-3.00# cat random_arg
40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b06061f1e4a76e3ced152855f7f5047 89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282 637ff2c7ad50c5f2018f7a865d646667 35bcc0568f9b8b8902fdc4f73fbadb49 34ce610edd36d834c08115de0c8e4568
<snip>
 


 And execute it
bash-3.00# ./a.out `cat random_arg `&
[1] 1536
Lets check what I can see with ps
bash-3.00# ps -efl|grep a.out
 0 S     root  1539   749   0  40 20        ?    347        ? 07:40:11 pts/1       0:00 grep a.out
 0 R     root  1536   749  70  89 20        ?    336          07:39:46 pts/1       0:26 ./a.out 40e4b176f31e662ae95ac93e995
 
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
    if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
        then echo OK $CNT
        else echo $i
    fi
    CNT=`echo $CNT + 1|bc`
done
 
 and exec
bash-3.00# ./z3.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
 
As you can see Solaris show me only two arguments with SYSV ps, but let me try with pargs

bash-3.00# pargs 1536
1536:   ./a.out 40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b060
argv[0]: ./a.out
argv[1]: 40e4b176f31e662ae95ac93e995cf4a4
argv[2]: f1e1c978a8007aa88bfffd675af2d56b
<snip>
argv[256]: 85fa98d8bf33f2b0507ec9714472d17f
OK, thats good, but I should know the PID :)



Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
    if [ `pgrep -f $i|wc -l` -eq 1 ]
        then echo OK $CNT
        else echo $i
    fi
    CNT=`echo $CNT + 1|bc`
done

bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>

Ha, I can see only two arguments, this is too small for me
And sample check if this depend of number of variables in environment
bash-3.00# env|wc -l
      14
bash-3.00# export BB=1
bash-3.00# env|wc -l
      15
bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>

It seems do not depend and this is internal limitation
Let see what will happen with shorter arguments

bash-3.00# ./a.out 111 222 333 444 555 666 777 888 999&
[1] 2346
bash-3.00# cat z4.sh
CNT=1
for i in 111 222 333 444 555 666 777 888 999;
do
    if [ `pgrep -f $i|wc -l` -eq 1 ]
        then echo OK $CNT
        else echo $i
    fi
    CNT=`echo $CNT + 1|bc`
done
bash-3.00# ./z4.sh
OK 1
OK 2
OK 3
OK 4
OK 5
OK 6
OK 7
OK 8
OK 9
 

O, all seems OK, but why i cant reach more deeper in argument list. Let keep this question open :)

Beyond the limits - environment

1. Prerequisites
My environmnet is based on Solaris 10  10 9/10  with 640 MB of memory and CentOS 5.5 2.6.18-194.26.1.el5 with 512 MB, standard installation both of them run in VMWare environmnet. For shell I use bash (as common)

bash-3.00# bash --version
GNU bash, version 3.00.16(1)-release (i386-pc-solaris2.10)
Copyright (C) 2004 Free Software Foundation, Inc.


[root@centos ~]# bash --version
GNU bash, version 3.2.25(1)-release (i686-redhat-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.

All the tests will be done with default settings of the OS and environment

2. Test the maximum length variable
the script i will use is sample:

TEMPVAR=A
for i in {1..10000};
        do
        TEMPVAR=${TEMPVAR}$TEMPVAR;
        a=$(echo 2^$i+1|bc)
        b=$(echo $TEMPVAR|wc -c)
        if [ $a -ne $b ]
        then echo KO - $a, $b
        else echo $a
        fi
done
 2.1. And the execution in Linux


[root@centos ~]# ./z1.sh
3
5
<snip>
67108865
134217729
./z1.sh: xmalloc: cannot allocate 268435457 bytes (0 bytes allocated)
KO - 268435457, 0
[1]+  Stopped                 ./z1.sh
[root@centos ~]# vmstat 1 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0  10716 229500  25160 228524   66  139   138   178 1011  151 18 18 61  3  0
 0  0  10716 229500  25160 228524    0    0     0     0 1013   15  0  0 100  0  0
 As you can see above I get error allocating memory beyond the size of free memory
2.2. Execution in Solaris

bash-3.00# ./z1.sh
3
5
<snip>
8388609
16777217
./z1.sh: fork: Not enough space
./z1.sh: line 7: [: 33554433: unary operator expected
33554433
./z1.sh: fork: Not enough space
bash-3.00# vmstat 1 2
 kthr      memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr cd f0 s0 --   in   sy   cs us sy id
 0 0 0 856808 174100 50 1101 0  1  5  0 66  2 -0  3  0  336  801  308  9  7 84
 0 0 0 749832 70472   3  43  0  0  0  0  0 27  0  0  0  326  300  185  1  9 90
 
 The similar is situation in Solaris where i can allocate even less that half of free memory
Of course it is not very wise to use such variables, but who knows?
3. Number of commandline arguments
3.1. Sample C program
Next I will try to check what is the maximum of commandline arguments of one sample C program

#include <stdio.h>
int main(int argc, char *argv[])
{
        printf("%d\n", argc);
}

and the shell script I will use is

TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./a.out $TEMPVAR)
echo $a
done
 

3.1.1 On Linux

[root@centos ~]# cc z2.c
[root@centos ~]# ./a.out 1
2
[root@centos ~]# ./a.out 1 2
3
[root@centos ~]# ./z2.sh
3
5
<snip>
524289
1048577
./z2.sh: line 5: ./a.out: Argument list too long

 1m of arguments, that pretty much :)
3.1.2 Solaris

bash-3.00# /usr/sfw/bin/gcc z2.c
bash-3.00# ./a.out 1
2
bash-3.00# ./a.out 1 2
3
bash-3.00# ./z2.sh
3
5
<snip>
65537
131073
./z2.sh: line 5: ./a.out: Arg list too long
"Just" 128k, not so much as Linux, but anyway do someone will use so many for practical puspose?


3.2. Sample shell program
Next I will try to check what is the maximum of commandline arguments of one very sample shell program

echo $#

 using this shell script

TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./z2test.sh $TEMPVAR)
echo $a
done
 
 This time the results appear mich slower because of the invocation of subshell.
3.2.1 Linux
After 1 hour of waiting I stop the script, but before this I see in top

29827 root      25   0 47028  41m  476 R 99.9  8.3   2:38.86 bash

3.2.2. Solaris
The same (long waiting) happen in Solaris and after 128k i see in prstat

 15100 root       72M   69M run     20    0   0:09:01  96% bash/1

As conclusion 128K is the maximum practicle limit of number command line arguments

DNS - make it sample and fast, part 5 - split server

Split DNS or sometime named dual home DNS is a DNS server, connected to two or more networks and serve different information, depend of the network from where request is initiated. This is very helpful when you want to have different view, different IP addreses for the same server. For example your mail server will have one address when you access it from internal network and totally different when you access it from internet.
In this article I will use 10.1.0 and 10.0.1 networks as example
23. Define access control lists in bind configuration
Add in named.conf the lines
acl net1 { 10.1.0.0/24;};
acl net2 { 10.0.1.0/24;};

24. Redefine example.net zone to ne int.example.net
this
zone "example.net" in{
  type master;
  file "master.example.net";
  allow-transfer {11.1.0.10;};
};

become
zone "int.example.net" in{
  type master;
  file "master.example.net";
  allow-transfer {11.1.0.10;};
  allow-query { net1; };
};

This should be done because is not possible to set one domain to be defined twice
25. And the same for IP network
this
zone "0.1.10.IN-ADDR.ARPA" in{
  type master;
  file "10.1.0.rev";
  allow-transfer {10.1.0.10;};
};
become
zone "0.1.10.IN-ADDR.ARPA" in{
  type master;
  file "10.1.0.rev";
  allow-transfer {10.1.0.10;};
  allow-query { net1; };
};

26. Lets add new sections for external view of domain and IP addresses
zone "example.net.ext" in{
  type master;
  file "master.example.net.ext";
  allow-query { net1; net2; };
};
zone "1.0.10.IN-ADDR.ARPA" in{
  type master;
  file "10.0.1.rev";
  allow-query { net1;net2; };
};

27. And create the appropriate files
[root@nsd named]# cat master.example.net.ext
@ IN SOA ns.example.net. root@ns.example.net. (
        2010081501 ; serial
        3600 ; refresh
        900 ; retry
        1209600 ; expire
        43200 ; default_ttl
)
@       IN NS   ns.example.net.

ns              IN A    10.0.1.5
[root@nsd named]# cat 10.0.1.rev
@ IN SOA ns.example.net. root@ns.example.net. (
        2010008151 ; serial
        3600 ; refresh
        900 ; retry
        1209600 ; expire
        43200 ; default_ttl
)
@       IN NS   ns.example.net

5       IN PTR  ns.example.net.

Of course on my old files I replace records *.example.net with *.int.example.net
28. And tell name server to reload configuration
[root@nsd named]# rndc reload
server reload successful

29. And of course make some tests
29.1. First from network 10.1.0
[root@nsd named]# nslookup
> nsd
Server:         10.1.0.5
Address:        10.1.0.5#53

Name:   nsd.int.example.net
Address: 10.1.0.5
> set q=ns
> int.example.net
Server:         10.1.0.5
Address:        10.1.0.5#53

example.net     nameserver = nsd.int.example.net.
example.net     nameserver = centos.int.example.net.
> ns.example.net
Server:         10.1.0.5
Address:        10.1.0.5#53

Name:   ns.example.net
Address: 10.0.1.5
> 10.0.1.5
Server:         10.1.0.5
Address:        10.1.0.5#53

5.1.0.10.in-addr.arpa   name = ns.example.net.
29.2. And from network 10.1.0
[root@c01-n1 ~]# ifconfig|grep -E "Ethernet|inet addr"
eth0      Link encap:Ethernet  HWaddr 00:0C:29:E1:57:32
          inet addr:10.0.1.101  Bcast:10.0.1.255  Mask:255.255.255.0
          inet addr:127.0.0.1  Mask:255.0.0.0
[root@c01-n1 ~]# nslookup
> ns
Server:         10.0.1.5
Address:        10.0.1.5#53

Name:   ns.example.net
Address: 10.0.1.5
> 10.0.1.5
Server:         10.0.1.5
Address:        10.0.1.5#53

5.1.0.10.in-addr.arpa   name = ns.example.net.
> nsd
Server:         10.0.1.5
Address:        10.0.1.5#53

** server can't find nsd: NXDOMAIN
> nsd.int.example.net
Server:         10.0.1.5
Address:        10.0.1.5#53

** server can't find nsd.int.example.net: NXDOMAIN
So if I name my mail server just mail and add in search domains int.example.net and example.net I will have access to the same server independently of the network I am attached. And I will give no access to the people outside to information about my internal network

DNS - make it sample and fast, part 4 - master/slave

12. Why is need master/slave architecture
There are many reasons why is need this architecture. First at all for redundancy, if you have only one DNS server what will happen in case of hardware failure? Or if DNS software or OS crash? Thats because it is wise to have more that one server. And to avoid synchronization efforts and mistakes will be good to do it automatically.
Next possible reason is spread the load of DNS requests. In case of big enterprise DNS requests can overload the server if there is only one. Of course you can replace it with new, more powerful server, but this will be (again) single point of failure.
Other reason can be security. If you expose to for public access only secondary server this will prevent (more or less) the possibility of harm on some way the resolv of hostnames/IP addreses. At the end this is only one server, so do not forget to implement all the security precautions you usually do
13. Configure to allow transfers to slave server(s)
In file /var/named/chroot/etc/named.conf the block
zone "example.net" in{
  type master;
  file "master.example.net";
};

will become
zone "example.net" in{
  type master;
  file "master.example.net";
  allow-transfer {10.1.0.10;};
};
and
zone "0.1.10.IN-ADDR.ARPA" in{
  type master;
  file "10.1.0.rev";
};
will become
zone "0.1.10.IN-ADDR.ARPA" in{
  type master;
  file "10.1.0.rev";
  allow-transfer {10.1.0.10;};
};

14. Configure slave server
The base is the same so lets edit /var/named/chroot/etc/named.conf. I will use config file from master and just edit some sections like
zone "example.net" in{
  type master;
  file "master.example.net";
  allow-transfer {10.1.0.10;};
};

become
zone "example.net" in{
  type slave;
  file "slave.example.net";
  masters {10.1.0.5;};
};
and
zone "0.1.10.IN-ADDR.ARPA" in{
  type master;
  file "10.1.0.rev";
  allow-transfer {10.1.0.10;};
};

become
zone "0.1.10.IN-ADDR.ARPA" in{
  type slave;
  file "10.1.0.rev.slave";
  masters {10.1.0.5;};
};

14. Check the configuration
[root@centos named]# service named configtest
zone localhost/IN: loaded serial 42
zone 0.0.127.in-addr.arpa/IN: loaded serial 1997022700

15. This look nice, but it is need to do few more steps
>/var/named/chroot/var/named/10.1.0.rev.slave
>/var/named/chroot/var/named/slave.example.net
chown named:named /var/named/chroot/var/named/10.1.0.rev.slave /var/named/chroot/var/named/slave.example.net
mkdir -p /var/named/chroot/tmp
chmod 1777 /var/named/chroot/tmp
last two lines are because of the chroot environment
16. Run the server
[root@centos named]# service named start
Starting named:                                            [  OK  ]
17. And of course make some tests
[root@centos named]# nslookup
> server 10.1.0.10
Default server: 10.1.0.10
Address: 10.1.0.10#53
> nsd
Server:         10.1.0.10
Address:        10.1.0.10#53

Name:   nsd.example.net
Address: 10.1.0.5
> nsd.example.net.
Server:         10.1.0.10
Address:        10.1.0.10#53

Name:   nsd.example.net
Address: 10.1.0.5
> centos
Server:         10.1.0.10
Address:        10.1.0.10#53

Name:   centos.example.net
Address: 10.1.0.10
> set q=mx
> example.net
Server:         10.1.0.10
Address:        10.1.0.10#53

example.net     mail exchanger = 1 centos.example.net.
> set q=ns
> example.net
Server:         10.1.0.10
Address:        10.1.0.10#53

example.net     nameserver = nsd.example.net.
> exit
18. All is fine except one minor mistake.
On the definitions for domain example,net is missing our new name server. Lets correct this
19. Go to master name server and edit /var/named/chroot/var/named/master.example.net
and line
        2010072501 ; serial

will be changed to
        2010080701 ; serial

and after line
@       IN NS   nsd.example.net.

will be added
@       IN NS   centos.example.net.

20. Make server reload the configuration
[root@nsd named]# service named reload
Reloading named:                                           [  OK  ]

21. And go back to slave server
[root@centos named]# nslookup
> server 10.1.0.10
Default server: 10.1.0.10
Address: 10.1.0.10#53
> set q=ns
> example.net
Server:         10.1.0.10
Address:        10.1.0.10#53

example.net     nameserver = centos.example.net.
example.net     nameserver = nsd.example.net.
22. E voila, we have already working master/slave DNS configuration

DNS - make it sample and fast, part 3

8. Master domain zone
I will create sample record for my master zone in file master.example.net
[root@nsd named]# cat master.example.net
@ IN SOA nsd.example.net. root@nsd.example.net. (
        2010072401 ; serial
        3600 ; refresh
        900 ; retry
        1209600 ; expire
        43200 ; default_ttl
)
@       IN NS   nsd.example.net.
@       IN MX 1 centos.example.net.

gw1        IN A    10.1.0.1
gw2         IN A    10.1.0.2
nsd         IN A    10.1.0.5
centos      IN A    10.1.0.10
As you can see I define one nameserver, one mail exchanger and four hosts
9. Reverse zone for network 10.1.0
@ IN SOA nsd.example.net. root@nsd.example.net. (
        2010072401 ; serial
        3600 ; refresh
        900 ; retry
        1209600 ; expire
        43200 ; default_ttl
)
@       IN NS   nsd.example.net

1       IN PTR gw1.example.net.
2       IN PTR gw1.example.net.
5       IN PTR nsd.example.net.
10      IN PTR centos.example.net.

I have only four IP addresses. Do not forget the dot on the end of host names
10. Check the configuration
[root@nsd named]# service named configtest
master.example.net:1: no TTL specified; using SOA MINTTL instead
zone example.net/IN: loaded serial 2010072401
zone localhost/IN: loaded serial 42
zone 0.0.127.in-addr.arpa/IN: loaded serial 1997022700
10.1.0.rev:1: no TTL specified; using SOA MINTTL instead
zone 0.1.10.IN-ADDR.ARPA/IN: loaded serial 2010072401

All seems OK, so its time to start the service and make some tests
11. Start and tests
[root@nsd named]# service named restart
Starting named:                                            [  OK  ]
[root@nsd named]# nslookup gw1
Server:         10.1.0.5
Address:        10.1.0.5#53

Name:   gw1.example.net
Address: 10.1.0.1

[root@nsd named]# nslookup gw1.example.net
Server:         10.1.0.5
Address:        10.1.0.5#53

Name:   gw1.example.net
Address: 10.1.0.1

[root@nsd named]# nslookup 10.1.0.5
Server:         10.1.0.5
Address:        10.1.0.5#53

5.0.1.10.in-addr.arpa   name = nsd.example.net.
[root@nsd named]# nslookup
> set q=ns
> example.net
Server:         10.1.0.5
Address:        10.1.0.5#53

example.net     nameserver = nsd.example.net.
> set q=mx
> example.net
Server:         10.1.0.5
Address:        10.1.0.5#53

example.net     mail exchanger = 1 centos.example.net.
> exit
At the end our nameserver is up and running and can server my domain and my IP range for my home network. For further information about parameters you see in usage in above files please refer official Bind documentation

DNS - make it sample and fast, part 2

6. Root servers
With one sample command I will create file for root server and will be sure this information is correct
[root@nsd chroot]# dig @e.root-servers.net . ns >/var/named/chroot/var/named/root.servers

And the content of the file is
[root@nsd chroot]# cat  /var/named/chroot/var/named/root.servers

; <<>> DiG 9.3.6-P1-RedHat-9.3.6-4.P1.el5_4.2 <<>> @e.root-servers.net . ns
; (1 server found)
;; global options:  printcmd
;; Got answer:
;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 35642
;; flags: qr aa rd; QUERY: 1, ANSWER: 13, AUTHORITY: 0, ADDITIONAL: 14

;; QUESTION SECTION:
;.                              IN      NS

;; ANSWER SECTION:
.                       518400  IN      NS      c.root-servers.net.
.                       518400  IN      NS      d.root-servers.net.
.                       518400  IN      NS      j.root-servers.net.
.                       518400  IN      NS      f.root-servers.net.
.                       518400  IN      NS      a.root-servers.net.
.                       518400  IN      NS      l.root-servers.net.
.                       518400  IN      NS      b.root-servers.net.
.                       518400  IN      NS      g.root-servers.net.
.                       518400  IN      NS      k.root-servers.net.
.                       518400  IN      NS      h.root-servers.net.
.                       518400  IN      NS      m.root-servers.net.
.                       518400  IN      NS      i.root-servers.net.
.                       518400  IN      NS      e.root-servers.net.

;; ADDITIONAL SECTION:
a.root-servers.net.     3600000 IN      A       198.41.0.4
a.root-servers.net.     3600000 IN      AAAA    2001:503:ba3e::2:30
b.root-servers.net.     3600000 IN      A       192.228.79.201
c.root-servers.net.     3600000 IN      A       192.33.4.12
d.root-servers.net.     3600000 IN      A       128.8.10.90
e.root-servers.net.     3600000 IN      A       192.203.230.10
f.root-servers.net.     3600000 IN      A       192.5.5.241
f.root-servers.net.     3600000 IN      AAAA    2001:500:2f::f
g.root-servers.net.     3600000 IN      A       192.112.36.4
h.root-servers.net.     3600000 IN      A       128.63.2.53
h.root-servers.net.     3600000 IN      AAAA    2001:500:1::803f:235
i.root-servers.net.     3600000 IN      A       192.36.148.17
i.root-servers.net.     3600000 IN      AAAA    2001:7fe::53
j.root-servers.net.     3600000 IN      A       192.58.128.30

;; Query time: 126 msec
;; SERVER: 192.203.230.10#53(192.203.230.10)
;; WHEN: Sat Jul 24 20:30:15 2010
;; MSG SIZE  rcvd: 500

But I will check just on case if all is OK with the file
[root@nsd chroot]# service named configtest
_default/example.net/in: file not found
_default/localhost/in: file not found
_default/0.0.127.in-addr.arpa/in: file not found
_default/0.1.10.IN-ADDR.ARPA/in: file not found
zone example.net/IN: loading master file master.example.net: file not found
zone localhost/IN: loading master file master.localhost: file not found
zone 0.0.127.in-addr.arpa/IN: loading master file localhost.rev: file not found
zone 0.1.10.IN-ADDR.ARPA/IN: loading master file 10.1.0.rev: file not found
Aha, no more complains about missing root servers file
7. Local domain and host files. (RFC-1912)
Because I am lazy I will get the files directly from bind package examples
[root@nsd chroot]# cp /usr/share/doc/bind-9.3.6/sample/var/named/named.local /var/named/chroot/var/named/localhost.rev
 [root@nsd chroot]# cp /usr/share/doc/bind-9.3.6/sample/var/named/localdomain.zone /var/named/chroot/var/named/master.locahost

And check what is the situation
[root@nsd chroot]# service named configtest
_default/example.net/in: file not found
_default/0.1.10.IN-ADDR.ARPA/in: file not found
zone example.net/IN: loading master file master.example.net: file not found
zone localhost/IN: loaded serial 42
zone 0.0.127.in-addr.arpa/IN: loaded serial 1997022700
zone 0.1.10.IN-ADDR.ARPA/IN: loading master file 10.1.0.rev: file not found

Cool, so part of the files are in place and I can continue with configuration

DNS - make it sample and fast

1. Preamble
In this article will try to show how to make sample DNS server. Do not expect here to see master/slave architecture, DNSSEC, sophisticated settings and so on (maybe in some late article). DNS server will serve example.net domain which is free and everyone can use it locally. Examples are based on Bind v9 and RHEL/CentOS. If you use different distribution, please consult your manuals.
2. First lets check prerequisites
[root@nsd ~]# rpm -qa |grep bind
bind-utils-9.3.6-4.P1.el5_4.2
ypbind-1.19-12.el5
bind-libs-9.3.6-4.P1.el5_4.2
bind-chroot-9.3.6-4.P1.el5_4.2
bind-9.3.6-4.P1.el5_4.2

If you do not have it installed you can do it with command
[root@nsd ~]# yum install bind-chroot

I will use chroot environment to make server a little bit more secure. But this do not change so much examples below
3. Check the directory tree
[root@nsd ~]# grep -v ^# /etc/sysconfig/named
ROOTDIR=/var/named/chroot
[root@nsd ~]# cd /var/named/chroot/
[root@nsd chroot]# tree
.
|-- dev
|   |-- null
|   |-- random
|   `-- zero
|-- etc
|   |-- localtime
|   `-- rndc.key
`-- var
    |-- log
    |-- named
    |   |-- data
    |   `-- slaves
    |-- run
    |   `-- named
    `-- tmp

10 directories, 5 files
4. Looks good so lets start with build our named.conf file
4.1. Options
options {
  directory "/var/named";
  allow-recursion {10.1.0.0/24;};
};

Base directory for files is set to be /var/named and access will be enabled from my local network IP range. Do not forget that the actual path is /var/named/chroot/var/named because of chroot environment
4.2. Root servers
zone "." {
  type hint;
  file "root.servers";
};

This is definition for servers, who will serve top level Internet zone . (dot)
4.3. Local host
zone "localhost" in{
  type master;
  file "master.localhost";
  allow-update{none;};
};
zone "0.0.127.in-addr.arpa" in{
  type master;
  file "localhost.rev";
  allow-update{none;};
};

Here is the definition for localhost zone and its corresponding reverse zone.
4.4. My zone
zone "example.net" in{
  type master;
  file "data/master.example.net";
};

I will not allow any slaves, because this will be the only DNS in to the network
4.5 My reverse zone
zone "0.1.10.IN-ADDR.ARPA" in{
  type master;
  file "10.1.0.rev";
};

I will manage locally 10.1.0 network
5. Create file named.conf and check configuration
[root@nsd chroot]# vi etc/named.conf
[root@nsd chroot]# service named configtest
_default/example.net/in: file not found
_default/localhost/in: file not found
_default/0.0.127.in-addr.arpa/in: file not found
_default/0.1.10.IN-ADDR.ARPA/in: file not found
zone example.net/IN: loading master file master.example.net: file not found
zone localhost/IN: loading master file master.localhost: file not found
zone 0.0.127.in-addr.arpa/IN: loading master file localhost.rev: file not found
zone 0.1.10.IN-ADDR.ARPA/IN: loading master file 10.1.0.rev: file not found

As it is expected zone files do not exist, but I will create them soon. Do not forget the actual place of config file is in chroot-ed environment i.e. /var/named/chroot/etc and not in /etc

Audit - something you have, something you do not know, part 2

5. Add some monitored files and directories
I will add monitoring of /var/run directory for read and write actions, /etc/rc.d/init.d for write and execution and /etc/shadow file for attribute change and write
[root@centos ~]# auditctl -w /etc/shadow -p wa -k shadow-file
[root@centos ~]# auditctl -w /var/run/ -p rw -k var-run-pids
[root@centos ~]# auditctl -w /etc/rc.d/init.d -p xw -k init-scripts

and check just in case the rules i define
[root@centos ~]# auditctl -l
LIST_RULES: exit,always dir=/var/run (0x8) perm=rw key=var-run-pids
LIST_RULES: exit,always watch=/etc/shadow perm=wa key=shadow-file
LIST_RULES: exit,always dir=/etc/rc.d/init.d (0x10) perm=wx key=init-scripts

6. Do some actions which will activate some audit trails
6.1. Create user (do change) in /etc/shadow file
[root@centos ~]# useradd test
[root@centos ~]# id test
uid=501(test) gid=501(test) groups=501(test)
[root@centos ~]# passwd test
Changing password for user test.
New UNIX password:
Retype new UNIX password:
passwd: all authentication tokens updated successfully
.
6.2. Start httpd daemon to get trails for execution in /etc/rc.d/init.d and write in /var/run
[root@centos ~]#  /etc/rc.d./init.d/httpd restart
Stopping httpd:                                            [FAILED]
Starting httpd:                                            [  OK  ]

7. Checks of audit logs
7.1. General check
[root@centos ~]# tail -20 /var/log/audit/audit.log
type=CONFIG_CHANGE msg=audit(1279423171.091:97): auid=0 ses=1 op="updated rules" path="/etc/shadow" key="shadow-file" list=4 res=1
type=SYSCALL msg=audit(1279423171.091:98): arch=40000003 syscall=38 success=yes exit=0 a0=b21317 a1=b211d6 a2=b235e4 a3=0 items=5 ppid=1921 pid=2034 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="passwd" exe="/usr/bin/passwd" key="shadow-file"
type=CWD msg=audit(1279423171.091:98):  cwd="/root"
type=PATH msg=audit(1279423171.091:98): item=0 name="/etc/" inode=229377 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423171.091:98): item=1 name="/etc/" inode=229377 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423171.091:98): item=2 name="/etc/nshadow" inode=229819 dev=fd:00 mode=0100400 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423171.091:98): item=3 name="/etc/shadow" inode=230260 dev=fd:00 mode=0100400 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423171.091:98): item=4 name="/etc/shadow" inode=229819 dev=fd:00 mode=0100400 ouid=0 ogid=0 rdev=00:00
type=USER_CHAUTHTOK msg=audit(1279423171.103:99): user pid=2034 uid=0 auid=0 msg='PAM: chauthtok acct="test" : exe="/usr/bin/passwd" (hostname=?, addr=?, terminal=pts/0 res=success)'
type=USER_CHAUTHTOK msg=audit(1279423171.106:100): user pid=2034 uid=0 auid=0 msg='op=change password id=501 exe="/usr/bin/passwd" (hostname=?, addr=?, terminal=pts/0 res=success)'
type=SYSCALL msg=audit(1279423197.516:101): arch=40000003 syscall=10 success=no exit=-2 a0=bfde9f8a a1=0 a2=805287c a3=bfde8594 items=1 ppid=2040 pid=2043 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="rm" exe="/bin/rm" key="var-run-pids"
type=CWD msg=audit(1279423197.516:101):  cwd="/"
type=PATH msg=audit(1279423197.516:101): item=0 name="/var/run/httpd.pid" inode=426024 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=SYSCALL msg=audit(1279423197.536:102): arch=40000003 syscall=10 success=no exit=-2 a0=bfd3bf8a a1=0 a2=805287c a3=bfd3bd24 items=1 ppid=2040 pid=2044 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="rm" exe="/bin/rm" key="var-run-pids"
type=CWD msg=audit(1279423197.536:102):  cwd="/"
type=PATH msg=audit(1279423197.536:102): item=0 name="/var/run/httpd.pid" inode=426024 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=SYSCALL msg=audit(1279423199.900:103): arch=40000003 syscall=5 success=yes exit=15 a0=82c4aa0 a1=8241 a2=1a4 a3=8241 items=2 ppid=1 pid=2048 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="httpd" exe="/usr/sbin/httpd" key="var-run-pids"
type=CWD msg=audit(1279423199.900:103):  cwd="/"
type=PATH msg=audit(1279423199.900:103): item=0 name="/etc/httpd/run/" inode=426024 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423199.900:103): item=1 name="/etc/httpd/run/httpd.pid" inode=426093 dev=fd:00 mode=0100644 ouid=0 ogid=0 rdev=00:00

7.2. Check for trails of actions, related to shadow file
[root@centos ~]# ausearch -f /etc/shadow
----
time->Sun Jul 18 05:19:21 2010
type=PATH msg=audit(1279423161.665:95): item=4 name="/etc/shadow" inode=230260 dev=fd:00 mode=0100400 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423161.665:95): item=3 name="/etc/shadow" inode=230225 dev=fd:00 mode=0100400 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423161.665:95): item=2 name="/etc/shadow+" inode=230260 dev=fd:00 mode=0100400 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423161.665:95): item=1 name="/etc/" inode=229377 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423161.665:95): item=0 name="/etc/" inode=229377 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=CWD msg=audit(1279423161.665:95):  cwd="/root"
type=SYSCALL msg=audit(1279423161.665:95): arch=40000003 syscall=38 success=yes exit=0 a0=bfc58c08 a1=80593a0 a2=bfc57b40 a3=0 items=5 ppid=1921 pid=2028 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="useradd" exe="/usr/sbin/useradd" key="shadow-file"
----
time->Sun Jul 18 05:19:31 2010
type=PATH msg=audit(1279423171.091:98): item=4 name="/etc/shadow" inode=229819 dev=fd:00 mode=0100400 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423171.091:98): item=3 name="/etc/shadow" inode=230260 dev=fd:00 mode=0100400 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423171.091:98): item=2 name="/etc/nshadow" inode=229819 dev=fd:00 mode=0100400 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423171.091:98): item=1 name="/etc/" inode=229377 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423171.091:98): item=0 name="/etc/" inode=229377 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=CWD msg=audit(1279423171.091:98):  cwd="/root"
type=SYSCALL msg=audit(1279423171.091:98): arch=40000003 syscall=38 success=yes exit=0 a0=b21317 a1=b211d6 a2=b235e4 a3=0 items=5 ppid=1921 pid=2034 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="passwd" exe="/usr/bin/passwd" key="shadow-file"

As you can see very clear here i have information about the userid, command, directory from where is executed command and of course timestamp of the action

7.3. From the log is visible run of httpd, create file, named /var/run/httpd.pid and i can use this file to reveal the action behind creation
[root@centos ~]# ausearch -f /var/run/httpd.pid
----
time->Sun Jul 18 05:19:57 2010
type=PATH msg=audit(1279423197.536:102): item=0 name="/var/run/httpd.pid" inode=426024 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=CWD msg=audit(1279423197.536:102):  cwd="/"
type=SYSCALL msg=audit(1279423197.536:102): arch=40000003 syscall=10 success=no exit=-2 a0=bfd3bf8a a1=0 a2=805287c a3=bfd3bd24 items=1 ppid=2040 pid=2044 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="rm" exe="/bin/rm" key="var-run-pids"
----
time->Sun Jul 18 05:19:57 2010
type=PATH msg=audit(1279423197.516:101): item=0 name="/var/run/httpd.pid" inode=426024 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=CWD msg=audit(1279423197.516:101):  cwd="/"
type=SYSCALL msg=audit(1279423197.516:101): arch=40000003 syscall=10 success=no exit=-2 a0=bfde9f8a a1=0 a2=805287c a3=bfde8594 items=1 ppid=2040 pid=2043 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="rm" exe="/bin/rm" key="var-run-pids"

7.4. Because of the way httpd run i see actual place of pid file is /etc/httpd/run/httpd.pid so lets check for this file too
[root@centos ~]# ausearch -f /etc/httpd/run/httpd.pid
----
time->Sun Jul 18 05:19:59 2010
type=PATH msg=audit(1279423199.900:103): item=1 name="/etc/httpd/run/httpd.pid" inode=426093 dev=fd:00 mode=0100644 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279423199.900:103): item=0 name="/etc/httpd/run/" inode=426024 dev=fd:00 mode=040755 ouid=0 ogid=0 rdev=00:00
type=CWD msg=audit(1279423199.900:103):  cwd="/"
type=SYSCALL msg=audit(1279423199.900:103): arch=40000003 syscall=5 success=yes exit=15 a0=82c4aa0 a1=8241 a2=1a4 a3=8241 items=2 ppid=1 pid=2048 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=(none) ses=1 comm="httpd" exe="/usr/sbin/httpd" key="var-run-pids"

7.5. Aha, so here is command to run /usr/sbin/httpd. I will check for run of some init script
[root@centos ~]# ausearch -k init-scripts
----
time->Sun Jul 18 05:34:19 2010
type=CONFIG_CHANGE msg=audit(1279424059.616:109): auid=0 op=add rule key="init-scripts" list=4 res=1
----
time->Sun Jul 18 05:35:33 2010
type=PATH msg=audit(1279424133.097:110): item=2 name=(null) inode=32807 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279424133.097:110): item=1 name=(null) inode=196644 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00
type=PATH msg=audit(1279424133.097:110): item=0 name="/etc/init.d/httpd" inode=230082 dev=fd:00 mode=0100755 ouid=0 ogid=0 rdev=00:00
type=CWD msg=audit(1279424133.097:110):  cwd="/"
type=EXECVE msg=audit(1279424133.097:110): argc=2 a0="/bin/bash" a1="/etc/init.d/httpd"
type=EXECVE msg=audit(1279424133.097:110): argc=3 a0="/bin/bash" a1="/etc/init.d/httpd" a2="restart"
type=SYSCALL msg=audit(1279424133.097:110): arch=40000003 syscall=11 success=yes exit=0 a0=bfafde5a a1=bfafd558 a2=94a4858 a3=5 items=3 ppid=2190 pid=2195 auid=0 uid=0 gid=0 euid=0 suid=0 fsuid=0 egid=0 sgid=0 fsgid=0 tty=pts0 ses=1 comm="httpd" exe="/bin/bash" key="init-scripts"
8. Est, voila, we reveal very detailed output from the actions, related to the areas/directories/files we think are sensitive and critical for our system

9. And at the end lets get summary report of audit sybsystem
[root@centos ~]# aureport

Summary Report
======================
Range of time in logs: 05/22/2010 11:01:19.717 - 07/18/2010 05:35:35.467
Selected time for report: 05/22/2010 11:01:19 - 07/18/2010 05:35:35.467
Number of changes in configuration: 56
Number of changes to accounts, groups, or roles: 21
Number of logins: 30
Number of failed logins: 21
Number of authentications: 12
Number of failed authentications: 1
Number of users: 1
Number of terminals: 11
Number of host names: 7
Number of executables: 12
Number of files: 45
Number of AVC's: 1
Number of MAC events: 1
Number of failed syscalls: 76
Number of anomaly events: 0
Number of responses to anomaly events: 0
Number of crypto events: 0
Number of keys: 4
Number of process IDs: 123
Number of events: 540

Of course in this short article it is not possible to show all the options we can use for audit one Linux system. For more detailed information please refer your documentation

Audit - something you have, something you do not know, part 1

Audit subsystem is installed usually on each Linux, but very often not used. In this article will try to show you what you can do with it. Be aware here will be used CentOS/RHEL, so the commands and files in your Linux may differ and you should consult documentation
1. Instalation. Lets check if the package is installed (usually it is)
[root@centos ~]# rpm -q audit
audit-1.7.17-3.el5

In this case the package is installed, but if you do not have it you can install with command
you install audit

Here are prerequisites for the package (nothing unusual)
[root@centos ~]# rpm -qR audit
/bin/bash
/bin/sh
/bin/sh
/bin/sh
audit-libs = 1.7.17-3.el5
chkconfig
config(audit) = 1.7.17-3.el5
coreutils
libaudit.so.0
libauparse.so.0
libc.so.6
libc.so.6(GLIBC_2.0)
libc.so.6(GLIBC_2.1)
libc.so.6(GLIBC_2.1.3)
libc.so.6(GLIBC_2.2)
libc.so.6(GLIBC_2.3)
libc.so.6(GLIBC_2.3.2)
libc.so.6(GLIBC_2.3.4)
libc.so.6(GLIBC_2.4)
libm.so.6
libm.so.6(GLIBC_2.0)
libnsl.so.1
libpthread.so.0
libpthread.so.0(GLIBC_2.0)
libpthread.so.0(GLIBC_2.1)
libpthread.so.0(GLIBC_2.3.2)
librt.so.1
libwrap.so.0
rpmlib(CompressedFileNames) <= 3.0.4-1
rpmlib(PayloadFilesHavePrefix) <= 4.0-1
rtld(GNU_HASH)

2. Next step is to check if audit daemon is started
[root@centos ~]# service auditd status
auditd (pid  1432) is running...

if it is not you can start it with
[root@centos ~]# service auditd start
Starting auditd:
[ OK ]
3. Configuration. One important point is to decide do you want to monitor system call or not. I will set to yes setting in file /etc/sysconfig/auditd will add line (or change if its exist)
AUDITD_DISABLE_CONTEXTS="no"

and restart the daemon
Next we should set where will be located audit log file. This is done in /etc/audit/auditd.conf with parameter log_file in format
log_file = /path/file

for example in my config this is
[root@centos ~]# grep ^log_file  /etc/audit/auditd.conf
log_file = /var/log/audit/audit.log

In some high security environments you will want to create dedicated filesystem for audit log files and you should care if this filesystem become full. This is done by parameter in /etc/audit/auditd.conf file
disk_full_action

This parameter can have values: ignore, syslog, exec, suspend, single, and halt
ignore - do nothing
syslog - write warning to syslog
exec /path/to/script - execute script
suspend - stop audit daemon for writing to disk
single - put server in single user mode
halt - shutdown the server
Of course there are many other parameters,but i just touch the most important (in my understanding)

4. Check the current situation
4.1. List current rules
[root@centos ~]# auditctl -l
No rules
4.2. Check current status
[root@centos ~]# auditctl -s
AUDIT_STATUS: enabled=1 flag=1 pid=2062 rate_limit=0 backlog_limit=320 lost=0 backlog=0

whodo on Linux way

Many times I become angry because of the absence of one UNIX command in Linux - whodo. So I try to write sample script to imitate it. See below:

for i in `who|tail -3|awk '{print $1}'`
do
who|grep -w $i
pstree -pu $i
done

I know the output is different, but for my personal purpose this is enough :)

Dealing with variables in shell environment - canonical way

One of the very important elements of interactive work with UNIX/Linux is a shell, especially variables in shell environment. To work with variables sometime can be easy, sometime hard. So here I will try to show some of the ideas how to deal with them.

1. how to see the variable: use one of the commands:

set|grep VAR
or

echo $VAR

2. how to set variable:

VAR=value;export VAR

or if you want to add some value to the existing in the variable

VAR=$VAR:value;export VAR

Of course export is need only in case you want to populate value to subinstrances of current shell. I use this way because it work on bourne shell, korn shell and bash. If you are sure script will work only in bash and/or korn shell use short form:

export VAR=value

3. how to destroy the variable:

unset VAR

I can use

VAR="";export VAR

but this only will "zero" the content and not destroy the variable

4. how to change (inside) the content of variable. You can make it entering the entire string (or number), but this little script will help you do it on the fly. Do not forget script is just an example and you should "tune" it to do the real work

VAR=`echo $VAR|sed s/string//g`;export VAR

Shell in nutshell or little tips and tricks

Today I will try to show  some tricks in UNIX/Linux shells maybe you do not know:

- use . (dot) to execute command or script in the current instance of the shell, this help much when you try to set some variables or change to directory with long and complicated path

- use - (dash) to go to previous working directory. this do not work in bourne shell

- use ${n..nn} in bash to generate sequence of numbers. Maybe was something similar in korn shell, but I can't remember.

$ printf "%03d " {1..100}

Also you can use seq program (mainly available in Linux)

$ printf "%03d " `seq 1 100`

- use bc to make arithmetic calculations on the fly:

# echo 2^15|bc

32768

- printf is very helpfull when you try to create well formatted output. Just follow the same format as in C

To be continued (maybe)...

My best friend - cron

I love cron, but there are few details missed from most of the users:

1. It's wise to set only one command to be executed, do not play with one long line, just a little script. This will help to avoid errors and problems.

2. Force set environment variables you need for script. Script run by cron have very limited set of variables. Better add as first line of script:

. /my/home/.profile
or startup file for your shell (.bash_profile for example if you use bash as default shell)

3. As first line in your script set shell need for execution, do not rely on default one. This is named shebang and here you can find very good explanation what is this and how to use it.

4. Do not play with tricky sets of minutes, hours, etc. Use full citation, this will work on all UNIXes and Linux and give much better visibility:

0,10,20,30,40,50 * * * * /path/myscript

5. Be careful when you use the days of week.  Beside other time variables which "use" AND as logical function between them Day of week use OR to the day of month.

Tar - the game continue

Sometime, some sysadmins forget one little detail about UNIX tar and make archives with leading slash. This can be helpful when you try to restore fast some files or directories, but in most of the cases is very bad habit. In many cases it's just need to extract somewhere (but not on the original place) file and compare it with the current file. So you have archive with leading slash and want to extract something. Let do it on a little complicated, but safe way:

mkdir -p /tmp/lib
for i in `ldd /usr/bin/tar|awk  '{print $3}'`;
do
cp $i /tmp/lib
done
cp /usr/bin/tar /tmp
mkdir -p /tmp/usr/lib
cp /usr/lib/ld.so.1 /tmp/usr/lib
chroot /tmp ./tar xvf archive.tar

In the man page, I get the idea, (SUN Solaris chroot) was mentioned full list of so libraries, but I am lazy and hate to write. And want to avoid mistakes. BTW in the documentation is not mentioned ld.so library and without this library tar do not want to run :-)

P.S. The script was tested only on Solaris 10, but (depend of format of ldd) will work on other UNIXes and Linux
P.P.S. You can use pax or GNU tar but as they are not a standard in UNIX world I try to make my solution universal

Playing with tar

Every sysadmin know that tar program do not accept wildcards for extract files so if you want to extract only one or few files you should play with construction like this:
 tar xvf archive.tar `tar tvf archive.tar|grep string` 
But on complex requests will be need to do it several times and if archive reside on tape (do not forget from where come the name of utility tar) this can take long time. Instead is much better to deal with files:
tar tvf archive.tar>/tmp/somefilename 
make some filtering on the listing
tar xvf archive.tar `cat /tmp/somefilename` 
P.S. I use cat to make command work independently (almost) of version of tar, UNIX flavour or Linux

Play with compressed tar files

Time to time every system administrator or user should extract file or entire archive from compressed tar, but quite often in filesystems there is not enough free space. In this case it's possible to extract file(s) on the fly without first uncompress and then untar:

gzip -dc archive.tar.gz|tar xf -

The above line can be modified when you should deal with broken archives. If you try to decompress such archive gzip just stop when find error in compression. But you want to save what's possible. So let's make it on this way:

cat archive.tar.gz|gzip -dc|tar xf -

I know it is possible to use GNU tar to extract files from compressed archive, but in UNIX world you can't be sure this software is installed :-)

Few remarks:

1. It's wise to use 'f -' key because in some Unixes without explicit 'f' tar will use tape device or whatever is defiined in TAPE variable
2. In above examples beside less diskpace usage you benefit from parallel execution (from pipe)

Welcome

Welcome to my technical blog. Here I will try to provide little tips, tricks and ideas, related to UNIX/Linux. And here is the first: Use this code to get size of directories. This will help you easy find where diskspace disappear:
du -sk /directory/*
Here is one example
du -sk /var/*
0       /var/account
0       /var/adm
149752  /var/cache
0       /var/crash
8       /var/db
0       /var/empty
0       /var/games
0       /var/gopher
0       /var/kerberos
208928  /var/lib
0       /var/local
0       /var/lock
5868    /var/log
0       /var/mail
0       /var/nis
0       /var/opt
0       /var/preserve
0       /var/run
4       /var/spool
0       /var/target
179572  /var/tmp
But we should browse a lot so let sort the output by size
du -sk /var/*|sort -g
0       /var/account
0       /var/adm
0       /var/crash
0       /var/empty
0       /var/games
0       /var/gopher
0       /var/kerberos
0       /var/local
0       /var/lock
0       /var/mail
0       /var/nis
0       /var/opt       
0       /var/preserve  
0       /var/run       
0       /var/target    
0       /var/yp        
4       /var/spool     
8       /var/db        
5868    /var/log       
149752  /var/cache     
179572  /var/tmp       
208928  /var/lib   
And we do not need so much lines, so let get only few last
du -sk /var/*|sort -g|tail -5                       
8       /var/db
5872    /var/log
149752  /var/cache
179572  /var/tmp
208928  /var/lib
E voila, we have candidate directories to search for space eater