Comment from Paul for the article make me rerun the tests in different conditions: I update the memory of both machines to 1024 MB and use scripts (see below)
[root@centos ~]# cat z1.sh
TEMPVAR=A
for i in {1..1000};
do
TEMPVAR=${TEMPVAR}$TEMPVAR;
a=$(echo 2^$i|bc)
b=${#TEMPVAR}
if [ $a -ne $b ]
then echo KO - $a, $b
else echo $a
fi
done
[root@centos ~]# cat z1.1.sh
TEMPVAR=A
for i in {1..1000};
do
TEMPVAR=${TEMPVAR}$TEMPVAR;
a=$(echo 2^$i+1|bc)
b=$(echo $TEMPVAR|wc -c)
if [ $a -ne $b ]
then echo KO - $a, $b
else echo $a
fi
done
Solaris
On Solaris independently of the amout of memory and the way of get length of environment variable the result is same:
bash-3.00# vmstat 1 2
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr cd f0 s0 -- in sy cs us sy id
0 0 0 1075080 616428 152 3085 0 0 0 0 204 10 -0 3 0 309 3345 1511 14 19 67
1 0 0 1407772 753456 18 50 0 0 0 0 0 0 0 0 0 304 324 142 0 1 99
It seems 32MB is internal limitation in bash (or OS) in Solaris
bash-3.00# ./z1.sh
2
4
<snip>
8388608
16777216
33554432
./z1.sh: fork: Not enough space
./z1.sh: line 7: [: -ne: unary operator expected
./z1.sh: fork: Not enough space
bash-3.00# ./z1.1.sh
3
5
<snip>
16777217
33554433
./z1.1.sh: fork: Not enough space
./z1.1.sh: fork: Not enough space
KO - ,
./z1.1.sh: fork: Not enough space
And the execution times follow the common sense
bash-3.00# time ./z1.sh
2
4
<snip>
16777216
33554432
real 0m9.670s
user 0m6.454s
sys 0m3.014s
bash-3.00# time ./z1.1.sh
3
5
<snip>
16777217
33554433
real 0m13.709s
user 0m8.040s
sys 0m5.352s
Linux
But on linux
[root@centos ~]# vmstat 1 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 10416 1016704 252 4072 75 804 318 828 1008 77 19 44 32 4 0
0 0 10416 1016704 252 4096 0 0 0 0 1015 14 0 1 99 0 0
my method of count permit me to reach bigger length
[root@centos ~]# ./z1.sh
2
4
<snip>
67108864
134217728
./z1.sh: xmalloc: cannot allocate 536870913 bytes (0 bytes allocated)
[root@centos ~]# ./z1.1.sh
3
5
<snip>
134217729
268435457
./z1.1.sh: xmalloc: cannot allocate 1073741825 bytes (0 bytes allocated)
KO - 536870913, 0
./z1.1.sh: xmalloc: cannot allocate 1073741825 bytes (0 bytes allocated)
and need less execution time (with limit to 2^26)
[root@centos ~]# time ./z1.sh
2
4
<snip>
33554432
67108864
real 1m58.730s
user 0m33.084s
sys 1m27.562s
[root@centos ~]# time ./z1.1.sh
3
5
<snip>
33554433
67108865
real 1m36.457s
user 0m50.206s
sys 0m47.815s
Final conclusion: this need further investigations :)
Beyond the limits - environment (cont)
4. Accessable command line argument
Now I will try to see how many are acceptable command line arguments from system utils like pgrep, pkill and similar. For this puspose I will use dummy C program which do nothing
#include <stdio.h>
int main(int argc, char *argv[])
{
while(1){};
}
And will generate sequence of random numbers to be fure will have ho false results
for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|md5sum|awk '{printf $1" "}'>>random_arg;done
[root@centos ~]# cat random_arg
d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09
<snip>
1187d29932b7d5827080684f16205330 92398e4719194f13a1a79064c5a69d05 e1cd425a036f77cc6fb6f3002d00bada a0c6d3b48c5e8146b2fe0b5f74383b9c
Let try with shorter arguments
4.1. Linux
Compile the program
[root@centos ~]# cc z4.c
And execute it
[root@centos ~]# ./a.out `cat random_arg `&
[1] 5094
Lets check what I can see with ps
[[root@centos ~]# ps -efl|grep a.out
0 R root 5094 2174 99 85 0 - 403 - 06:13 pts/0 00:00:35 ./a.out d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09 73e522501878712898d1c47342b9bd66
<snip>
f566402f18f3ebe15453d3a74fc1c534 f87fe9ab1f5f85238ce01ca5d62d57e1 d05d9d3b9cb608d8a177434a4cda5c69 43e80e6069a763a0da17abdba8a06b30 d79a21984d368e4b574e4690a749c7af f5c9eddc01dd1a98bdb7ecbd5eba87a7 249a93f99bb66ff2361f722d1de2b
0 R root 5101 2174 0 78 0 - 1001 - 06:13 pts/0 00:00:00 grep a.out
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
and exec
[root@centos ~]# ./z3.sh
OK 1
OK 2
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
So I can see only 123 arguments. Hm, this a big number, but no so much
Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
The same result. So i can see beyond parameter number 123. And sample chech if this depend of number of variables in environment
[root@centos ~]# env|wc -l
20
[root@centos ~]# export BB=1
[root@centos ~]# env|wc -l
21
[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
It seems do not depend and this is internal limitation
4.2. Solaris
Compile the program
bash-3.00# /usr/sfw/bin/gcc z3.c
and generate random numbers
for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|digest -a md5|awk '{printf $1" "}'>>random_arg;done
bash-3.00# cat random_arg
40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b06061f1e4a76e3ced152855f7f5047 89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282 637ff2c7ad50c5f2018f7a865d646667 35bcc0568f9b8b8902fdc4f73fbadb49 34ce610edd36d834c08115de0c8e4568
<snip>
And execute it
bash-3.00# ./a.out `cat random_arg `&
[1] 1536
Lets check what I can see with ps
bash-3.00# ps -efl|grep a.out
0 S root 1539 749 0 40 20 ? 347 ? 07:40:11 pts/1 0:00 grep a.out
0 R root 1536 749 70 89 20 ? 336 07:39:46 pts/1 0:26 ./a.out 40e4b176f31e662ae95ac93e995
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
and exec
bash-3.00# ./z3.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
As you can see Solaris show me only two arguments with SYSV ps, but let me try with pargs
bash-3.00# pargs 1536
1536: ./a.out 40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b060
argv[0]: ./a.out
argv[1]: 40e4b176f31e662ae95ac93e995cf4a4
argv[2]: f1e1c978a8007aa88bfffd675af2d56b
<snip>
argv[256]: 85fa98d8bf33f2b0507ec9714472d17f
OK, thats good, but I should know the PID :)
Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
Ha, I can see only two arguments, this is too small for me
And sample check if this depend of number of variables in environment
bash-3.00# env|wc -l
14
bash-3.00# export BB=1
bash-3.00# env|wc -l
15
bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
It seems do not depend and this is internal limitation
Let see what will happen with shorter arguments
bash-3.00# ./a.out 111 222 333 444 555 666 777 888 999&
[1] 2346
bash-3.00# cat z4.sh
CNT=1
for i in 111 222 333 444 555 666 777 888 999;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
bash-3.00# ./z4.sh
OK 1
OK 2
OK 3
OK 4
OK 5
OK 6
OK 7
OK 8
OK 9
O, all seems OK, but why i cant reach more deeper in argument list. Let keep this question open :)
Now I will try to see how many are acceptable command line arguments from system utils like pgrep, pkill and similar. For this puspose I will use dummy C program which do nothing
#include <stdio.h>
int main(int argc, char *argv[])
{
while(1){};
}
And will generate sequence of random numbers to be fure will have ho false results
for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|md5sum|awk '{printf $1" "}'>>random_arg;done
[root@centos ~]# cat random_arg
d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09
<snip>
1187d29932b7d5827080684f16205330 92398e4719194f13a1a79064c5a69d05 e1cd425a036f77cc6fb6f3002d00bada a0c6d3b48c5e8146b2fe0b5f74383b9c
Let try with shorter arguments
4.1. Linux
Compile the program
[root@centos ~]# cc z4.c
And execute it
[root@centos ~]# ./a.out `cat random_arg `&
[1] 5094
Lets check what I can see with ps
[[root@centos ~]# ps -efl|grep a.out
0 R root 5094 2174 99 85 0 - 403 - 06:13 pts/0 00:00:35 ./a.out d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09 73e522501878712898d1c47342b9bd66
<snip>
f566402f18f3ebe15453d3a74fc1c534 f87fe9ab1f5f85238ce01ca5d62d57e1 d05d9d3b9cb608d8a177434a4cda5c69 43e80e6069a763a0da17abdba8a06b30 d79a21984d368e4b574e4690a749c7af f5c9eddc01dd1a98bdb7ecbd5eba87a7 249a93f99bb66ff2361f722d1de2b
0 R root 5101 2174 0 78 0 - 1001 - 06:13 pts/0 00:00:00 grep a.out
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
and exec
[root@centos ~]# ./z3.sh
OK 1
OK 2
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
So I can see only 123 arguments. Hm, this a big number, but no so much
Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
The same result. So i can see beyond parameter number 123. And sample chech if this depend of number of variables in environment
[root@centos ~]# env|wc -l
20
[root@centos ~]# export BB=1
[root@centos ~]# env|wc -l
21
[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
It seems do not depend and this is internal limitation
4.2. Solaris
Compile the program
bash-3.00# /usr/sfw/bin/gcc z3.c
and generate random numbers
for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|digest -a md5|awk '{printf $1" "}'>>random_arg;done
bash-3.00# cat random_arg
40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b06061f1e4a76e3ced152855f7f5047 89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282 637ff2c7ad50c5f2018f7a865d646667 35bcc0568f9b8b8902fdc4f73fbadb49 34ce610edd36d834c08115de0c8e4568
<snip>
And execute it
bash-3.00# ./a.out `cat random_arg `&
[1] 1536
Lets check what I can see with ps
bash-3.00# ps -efl|grep a.out
0 S root 1539 749 0 40 20 ? 347 ? 07:40:11 pts/1 0:00 grep a.out
0 R root 1536 749 70 89 20 ? 336 07:39:46 pts/1 0:26 ./a.out 40e4b176f31e662ae95ac93e995
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
and exec
bash-3.00# ./z3.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
As you can see Solaris show me only two arguments with SYSV ps, but let me try with pargs
bash-3.00# pargs 1536
1536: ./a.out 40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b060
argv[0]: ./a.out
argv[1]: 40e4b176f31e662ae95ac93e995cf4a4
argv[2]: f1e1c978a8007aa88bfffd675af2d56b
<snip>
argv[256]: 85fa98d8bf33f2b0507ec9714472d17f
OK, thats good, but I should know the PID :)
Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
Ha, I can see only two arguments, this is too small for me
And sample check if this depend of number of variables in environment
bash-3.00# env|wc -l
14
bash-3.00# export BB=1
bash-3.00# env|wc -l
15
bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
It seems do not depend and this is internal limitation
Let see what will happen with shorter arguments
bash-3.00# ./a.out 111 222 333 444 555 666 777 888 999&
[1] 2346
bash-3.00# cat z4.sh
CNT=1
for i in 111 222 333 444 555 666 777 888 999;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
bash-3.00# ./z4.sh
OK 1
OK 2
OK 3
OK 4
OK 5
OK 6
OK 7
OK 8
OK 9
O, all seems OK, but why i cant reach more deeper in argument list. Let keep this question open :)
Beyond the limits - environment
1. Prerequisites
My environmnet is based on Solaris 10 10 9/10 with 640 MB of memory and CentOS 5.5 2.6.18-194.26.1.el5 with 512 MB, standard installation both of them run in VMWare environmnet. For shell I use bash (as common)
bash-3.00# bash --version
GNU bash, version 3.00.16(1)-release (i386-pc-solaris2.10)
Copyright (C) 2004 Free Software Foundation, Inc.
[root@centos ~]# bash --version
GNU bash, version 3.2.25(1)-release (i686-redhat-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.
All the tests will be done with default settings of the OS and environment
2. Test the maximum length variable
the script i will use is sample:
TEMPVAR=A
for i in {1..10000};
do
TEMPVAR=${TEMPVAR}$TEMPVAR;
a=$(echo 2^$i+1|bc)
b=$(echo $TEMPVAR|wc -c)
if [ $a -ne $b ]
then echo KO - $a, $b
else echo $a
fi
done
2.1. And the execution in Linux
[root@centos ~]# ./z1.sh
3
5
<snip>
67108865
134217729
./z1.sh: xmalloc: cannot allocate 268435457 bytes (0 bytes allocated)
KO - 268435457, 0
[1]+ Stopped ./z1.sh
[root@centos ~]# vmstat 1 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 10716 229500 25160 228524 66 139 138 178 1011 151 18 18 61 3 0
0 0 10716 229500 25160 228524 0 0 0 0 1013 15 0 0 100 0 0
As you can see above I get error allocating memory beyond the size of free memory
2.2. Execution in Solaris
bash-3.00# ./z1.sh
3
5
<snip>
8388609
16777217
./z1.sh: fork: Not enough space
./z1.sh: line 7: [: 33554433: unary operator expected
33554433
./z1.sh: fork: Not enough space
bash-3.00# vmstat 1 2
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr cd f0 s0 -- in sy cs us sy id
0 0 0 856808 174100 50 1101 0 1 5 0 66 2 -0 3 0 336 801 308 9 7 84
0 0 0 749832 70472 3 43 0 0 0 0 0 27 0 0 0 326 300 185 1 9 90
The similar is situation in Solaris where i can allocate even less that half of free memory
Of course it is not very wise to use such variables, but who knows?
3. Number of commandline arguments
3.1. Sample C program
Next I will try to check what is the maximum of commandline arguments of one sample C program
#include <stdio.h>
int main(int argc, char *argv[])
{
printf("%d\n", argc);
}
and the shell script I will use is
TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./a.out $TEMPVAR)
echo $a
done
3.1.1 On Linux
[root@centos ~]# cc z2.c
[root@centos ~]# ./a.out 1
2
[root@centos ~]# ./a.out 1 2
3
[root@centos ~]# ./z2.sh
3
5
<snip>
524289
1048577
./z2.sh: line 5: ./a.out: Argument list too long
1m of arguments, that pretty much :)
3.1.2 Solaris
bash-3.00# /usr/sfw/bin/gcc z2.c
bash-3.00# ./a.out 1
2
bash-3.00# ./a.out 1 2
3
bash-3.00# ./z2.sh
3
5
<snip>
65537
131073
./z2.sh: line 5: ./a.out: Arg list too long
"Just" 128k, not so much as Linux, but anyway do someone will use so many for practical puspose?
3.2. Sample shell program
Next I will try to check what is the maximum of commandline arguments of one very sample shell program
echo $#
using this shell script
TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./z2test.sh $TEMPVAR)
echo $a
done
This time the results appear mich slower because of the invocation of subshell.
3.2.1 Linux
After 1 hour of waiting I stop the script, but before this I see in top
29827 root 25 0 47028 41m 476 R 99.9 8.3 2:38.86 bash
3.2.2. Solaris
The same (long waiting) happen in Solaris and after 128k i see in prstat
15100 root 72M 69M run 20 0 0:09:01 96% bash/1
As conclusion 128K is the maximum practicle limit of number command line arguments
My environmnet is based on Solaris 10 10 9/10 with 640 MB of memory and CentOS 5.5 2.6.18-194.26.1.el5 with 512 MB, standard installation both of them run in VMWare environmnet. For shell I use bash (as common)
bash-3.00# bash --version
GNU bash, version 3.00.16(1)-release (i386-pc-solaris2.10)
Copyright (C) 2004 Free Software Foundation, Inc.
[root@centos ~]# bash --version
GNU bash, version 3.2.25(1)-release (i686-redhat-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.
All the tests will be done with default settings of the OS and environment
2. Test the maximum length variable
the script i will use is sample:
TEMPVAR=A
for i in {1..10000};
do
TEMPVAR=${TEMPVAR}$TEMPVAR;
a=$(echo 2^$i+1|bc)
b=$(echo $TEMPVAR|wc -c)
if [ $a -ne $b ]
then echo KO - $a, $b
else echo $a
fi
done
2.1. And the execution in Linux
[root@centos ~]# ./z1.sh
3
5
<snip>
67108865
134217729
./z1.sh: xmalloc: cannot allocate 268435457 bytes (0 bytes allocated)
KO - 268435457, 0
[1]+ Stopped ./z1.sh
[root@centos ~]# vmstat 1 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 10716 229500 25160 228524 66 139 138 178 1011 151 18 18 61 3 0
0 0 10716 229500 25160 228524 0 0 0 0 1013 15 0 0 100 0 0
As you can see above I get error allocating memory beyond the size of free memory
2.2. Execution in Solaris
bash-3.00# ./z1.sh
3
5
<snip>
8388609
16777217
./z1.sh: fork: Not enough space
./z1.sh: line 7: [: 33554433: unary operator expected
33554433
./z1.sh: fork: Not enough space
bash-3.00# vmstat 1 2
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr cd f0 s0 -- in sy cs us sy id
0 0 0 856808 174100 50 1101 0 1 5 0 66 2 -0 3 0 336 801 308 9 7 84
0 0 0 749832 70472 3 43 0 0 0 0 0 27 0 0 0 326 300 185 1 9 90
The similar is situation in Solaris where i can allocate even less that half of free memory
Of course it is not very wise to use such variables, but who knows?
3. Number of commandline arguments
3.1. Sample C program
Next I will try to check what is the maximum of commandline arguments of one sample C program
#include <stdio.h>
int main(int argc, char *argv[])
{
printf("%d\n", argc);
}
and the shell script I will use is
TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./a.out $TEMPVAR)
echo $a
done
3.1.1 On Linux
[root@centos ~]# cc z2.c
[root@centos ~]# ./a.out 1
2
[root@centos ~]# ./a.out 1 2
3
[root@centos ~]# ./z2.sh
3
5
<snip>
524289
1048577
./z2.sh: line 5: ./a.out: Argument list too long
1m of arguments, that pretty much :)
3.1.2 Solaris
bash-3.00# /usr/sfw/bin/gcc z2.c
bash-3.00# ./a.out 1
2
bash-3.00# ./a.out 1 2
3
bash-3.00# ./z2.sh
3
5
<snip>
65537
131073
./z2.sh: line 5: ./a.out: Arg list too long
"Just" 128k, not so much as Linux, but anyway do someone will use so many for practical puspose?
3.2. Sample shell program
Next I will try to check what is the maximum of commandline arguments of one very sample shell program
echo $#
using this shell script
TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./z2test.sh $TEMPVAR)
echo $a
done
This time the results appear mich slower because of the invocation of subshell.
3.2.1 Linux
After 1 hour of waiting I stop the script, but before this I see in top
29827 root 25 0 47028 41m 476 R 99.9 8.3 2:38.86 bash
3.2.2. Solaris
The same (long waiting) happen in Solaris and after 128k i see in prstat
15100 root 72M 69M run 20 0 0:09:01 96% bash/1
As conclusion 128K is the maximum practicle limit of number command line arguments
Solaris x86 root filesystem mirroring
Preamble
This document is directed to give idea how to create mirroring of root filesystem in Solaris x86 with the help of Solaris Volume Manager. Here will be used already installed OS and all the work will be done without need of reinstall
Prerequisites
1. First we should be sure have two identical harddisks in the server
# format
Searching for disks...done
AVAILABLE DISK SELECTIONS:
0. c0d0 <DEFAULT cyl 1563 alt 2 hd 255 sec 63>
/pci@0,0/pci-ide@7,1/ide@0/cmdk@0,0
1. c0d1 <DEFAULT cyl 1563 alt 2 hd 255 sec 63>
/pci@0,0/pci-ide@7,1/ide@0/cmdk@1,0
Specify disk (enter its number): ^C
2. Its need to create small slice for metadb information (usually slice 7) like:
partition> p
Current partition table (unnamed):
Total disk cylinders available: 1563 + 2 (reserved cylinders)
Part Tag Flag Cylinders Size Blocks
0 root wm 518 - 1562 8.01GB (1045/0/0) 16787925
1 swap wu 3 - 133 1.00GB (131/0/0) 2104515
2 backup wm 0 - 1562 11.97GB (1563/0/0) 25109595
3 unassigned wm 0 0 (0/0/0) 0
4 unassigned wm 0 0 (0/0/0) 0
5 unassigned wm 0 0 (0/0/0) 0
6 unassigned wm 0 0 (0/0/0) 0
7 unassigned wm 134 - 135 15.69MB (2/0/0) 32130
8 boot wu 0 - 0 7.84MB (1/0/0) 16065
9 alternates wu 1 - 2 15.69MB (2/0/0) 32130
3. Next step is to create the same partitions on the second disk. To avoid human error it is much better to use some kind of automations:
prtvtoc /dev/rdsk/c0d0s2 > /tmp/c0d0s2.toc
fmthard -s /tmp/c0d0s2.toc /dev/rdsk/c0d1s2
4. Then we should identify the partitions need to be mirrored:
# egrep "ufs|swap" /etc/vfstab|grep "/dev/dsk"
/dev/dsk/c0d0s1 - - swap - no -
/dev/dsk/c0d0s0 /dev/rdsk/c0d0s0 / ufs 1 no -
LVM
1. Let’s create few copies of metadb on the partitions we create for this puspose:
# metadb -a -f -c 2 c0d0s7 c0d1s7
2. It’s time to put our disks where OS reside under management of SVM
# metainit -f d10 1 1 c0d0s0
d10: Concat/Stripe is setup
# metainit -f d11 1 1 c0d0s1
d11: Concat/Stripe is setup
# metainit d0 -m d10
d0: Mirror is setup
# metainit d1 -m d11
d1: Mirror is setup
# metaroot d0
3. Check newly created devices:
# ls -l /dev/md/rdsk
total 8
lrwxrwxrwx 1 root root 36 Aug 30 18:29 d0 -> ../../../devices/pseudo/md@0:0,0,raw
lrwxrwxrwx 1 root root 36 Aug 30 18:29 d1 -> ../../../devices/pseudo/md@0:0,1,raw
lrwxrwxrwx 1 root root 37 Aug 30 18:28 d10 -> ../../../devices/pseudo/md@0:0,10,raw
lrwxrwxrwx 1 root root 37 Aug 30 18:28 d11 -> ../../../devices/pseudo/md@0:0,11,raw
# ls -l /dev/md/dsk
total 8
lrwxrwxrwx 1 root root 36 Aug 30 18:29 d0 -> ../../../devices/pseudo/md@0:0,0,blk
lrwxrwxrwx 1 root root 36 Aug 30 18:29 d1 -> ../../../devices/pseudo/md@0:0,1,blk
lrwxrwxrwx 1 root root 37 Aug 30 18:28 d10 -> ../../../devices/pseudo/md@0:0,10,blk
lrwxrwxrwx 1 root root 37 Aug 30 18:28 d11 -> ../../../devices/pseudo/md@0:0,11,blk
4. Make appropriate changes in /etc/vfstab to get boot from mirror, not standard disks
# egrep "ufs|swap" /etc/vfstab|grep "/dev/md/dsk"
/dev/md/dsk/d1 - - swap - no -
/dev/md/dsk/d0 /dev/md/rdsk/d0 / ufs 1 no -
5. The next step is to flush cache buffers and reboot
# sync;sync;sync
# init 6
6. It is time to put second disk under management of SVN
# metainit -f d20 1 1 c0d1s0
d20: Concat/Stripe is setup
# metainit -f d21 1 1 c0d1s1
d21: Concat/Stripe is setup
7. And to add them to created previously mirrors. Be aware process of synchronisation will continue in background and you can check the process
# metattach d0 d20
d0: submirror d20 is attached
# metattach d1 d21
d1: submirror d21 is attached
8. Check the process of building mirrors and wait till they finnish
# metastat
d1: Mirror
Submirror 0: d11
State: Okay
Submirror 1: d21
State: Resyncing
Resync in progress: 96 % done
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 2104515 blocks (1.0 GB)
d11: Submirror of d1
State: Okay
Size: 2104515 blocks (1.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d0s1 0 No Okay Yes
d21: Submirror of d1
State: Resyncing
Size: 2104515 blocks (1.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d1s1 0 No Okay Yes
d0: Mirror
Submirror 0: d10
State: Okay
Submirror 1: d20
State: Resyncing
Resync in progress: 13 % done
Pass: 1
Read option: roundrobin (default)
Write option: parallel (default)
Size: 16787925 blocks (8.0 GB)
d10: Submirror of d0
State: Okay
Size: 16787925 blocks (8.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d0s0 0 No Okay Yes
d20: Submirror of d0
State: Resyncing
Size: 16787925 blocks (8.0 GB)
Stripe 0:
Device Start Block Dbase State Reloc Hot Spare
c0d1s0 0 No Okay Yes
Device Relocation Information:
Device Reloc Device ID
c0d1 Yes id1,cmdk@AVMware_Virtual_IDE_Hard_Drive=01000000000000000001
c0d0 Yes id1,cmdk@AVMware_Virtual_IDE_Hard_Drive=00000000000000000001
Boot
1. Next step is to check if partition on second disk is active
# fdisk /dev/rdsk/c0d1p0
Total disk size is 1566 cylinders
Cylinder size is 16065 (512 byte) blocks
Cylinders
Partition Status Type Start End Length %
= = = =
1 Active Solaris2 1 1565 1565 100
2. And to add boot record to the second disk to make it bootable
# /sbin/installgrub /boot/grub/stage1 /boot/grub/stage2 /dev/rdsk/c0d1s0
stage1 written to partition 0 sector 0 (abs 16065)
stage2 written to partition 0, 233 sectors starting at 50 (abs 16115)
3. Add new item in the boot menu (/boot/grub/menu.lst) to have alternative way to boot
title Alternate boot
root (hd1,0,a)
kernel /platform/i86pc/multiboot
module /platform/i86pc/boot_archive
4. Check if the new item is added to the boot menu
# bootadm list-menu
The location for the active GRUB menu is: /boot/grub/menu.lst
default 0
timeout 10
0 Solaris 10 5/08 s10x_u5wos_10 X86
1 Solaris failsafe
2 Alternate boot
5. That’s all, you have already mirrored root partition (plus swap)
Conclusion
In the document are not mentioned all the options and possibilities of SVM, but only short set, need to done the work. For further information, please consult official Oracle documentation.
Audit (Solaris) - something you have, something you do not know - part 1
Audit subsystem in Solaris is very powerful and can give you a lot of information. In this first article I will give you example how to play with audit for monitoring user actions
1. Check the user i want to monitor
# id test1
uid=200(test1) gid=1(other)
2. And the actions I want to record
# cat /etc/security/audit_class
#
.....
# File Format:
#
# mask:name:description
#
0x00000000:no:invalid class
...
0xffffffff:all:all classes (meta-class)
Of course in above file you will see much more classes and you can set to monitor and audit only part of them, but for this example I will use monitoring all
3. Set user and classes
# cat /etc/security/audit_user
#
...
# username:always:never
#
root:lo:no
test1:all:no
So here I will monitor everything and no exceptions
4. Run bsmconv because of first usage of audit
# /etc/security/bsmconv
This script is used to enable the Basic Security Module (BSM).
Shall we continue with the conversion now? [y/n] y
bsmconv: INFO: checking startup file.
bsmconv: INFO: turning on audit module.
bsmconv: INFO: initializing device allocation.
The Basic Security Module is ready.
If there were any errors, please fix them now.
Configure BSM by editing files located in /etc/security.
Reboot this system now to come up with BSM enabled.
5. And reboot
# init 6
updating /platform/i86pc/boot_archive
6. Login as test1 do some actions and exit
login as: test1
Password:
Last login: Sun Sep 26 10:13:00 2010 from 10.1.0.1
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
$ pwd
/export/home/test1
$ ls
$ df -k .
Filesystem kbytes used avail capacity Mounted on
/dev/dsk/c1t0d0s7 248647 1042 222741 1% /export/home
$ /usr/sbin/mkfile 10m aa
$ ls -l
total 20496
-rw------- 1 test1 other 10485760 Sep 26 10:13 aa
$ exit
7. Check the audit file(s)
# cd /var/audit
# ls -l
total 128
-rw------- 1 root root 64370 Sep 26 10:16 20100926081119.20100926081640.sol01
8. Browse the content of this audit file
# praudit 20100926081119.20100926081640.sol01
...
header,134,2,execve(2),,sol01,2010-09-26 10:13:35.371 +02:00
path,/usr/sbin/mkfile
attribute,100555,root,bin,3,853,0
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
header,102,2,open(2) - read,fe,sol01,2010-09-26 10:13:35.371 +02:00
path,/var/ld/ld.config
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,failure: No such file or directory,-1
header,114,2,munmap(2),,sol01,2010-09-26 10:13:35.371 +02:00
argument,1,0xfef8e000,addr
argument,2,0x10000,len
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
header,168,2,memcntl(2),,sol01,2010-09-26 10:13:35.371 +02:00
argument,1,0xfee80000,base
argument,2,0x1e4b0,len
argument,3,0x4,cmd
argument,4,0x3,arg
argument,5,0x0,attr
argument,6,0x0,mask
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
header,114,2,munmap(2),,sol01,2010-09-26 10:13:35.372 +02:00
argument,1,0xfefb0000,addr
argument,2,0x8000,len
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
header,139,2,open(2) - read,write,creat,trunc,,sol01,2010-09-26 10:13:35.386 +02:00
path,/export/home/test1/aa
attribute,100600,test1,other,7,5,0
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,3
header,150,2,close(2),,sol01,2010-09-26 10:13:35.429 +02:00
argument,1,0x3,fd
path,/export/home/test1/aa
attribute,100600,test1,other,7,5,0
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
...
As you can see clear here is executed command mkfile and created file, named aa in to the home directory of the user. Of course in above example I use only few of the features in the audit subsystem of Solaris, but in next article will try to present more in deep the options
1. Check the user i want to monitor
# id test1
uid=200(test1) gid=1(other)
2. And the actions I want to record
# cat /etc/security/audit_class
#
.....
# File Format:
#
# mask:name:description
#
0x00000000:no:invalid class
...
0xffffffff:all:all classes (meta-class)
Of course in above file you will see much more classes and you can set to monitor and audit only part of them, but for this example I will use monitoring all
3. Set user and classes
# cat /etc/security/audit_user
#
...
# username:always:never
#
root:lo:no
test1:all:no
So here I will monitor everything and no exceptions
4. Run bsmconv because of first usage of audit
# /etc/security/bsmconv
This script is used to enable the Basic Security Module (BSM).
Shall we continue with the conversion now? [y/n] y
bsmconv: INFO: checking startup file.
bsmconv: INFO: turning on audit module.
bsmconv: INFO: initializing device allocation.
The Basic Security Module is ready.
If there were any errors, please fix them now.
Configure BSM by editing files located in /etc/security.
Reboot this system now to come up with BSM enabled.
5. And reboot
# init 6
updating /platform/i86pc/boot_archive
6. Login as test1 do some actions and exit
login as: test1
Password:
Last login: Sun Sep 26 10:13:00 2010 from 10.1.0.1
Sun Microsystems Inc. SunOS 5.10 Generic January 2005
$ pwd
/export/home/test1
$ ls
$ df -k .
Filesystem kbytes used avail capacity Mounted on
/dev/dsk/c1t0d0s7 248647 1042 222741 1% /export/home
$ /usr/sbin/mkfile 10m aa
$ ls -l
total 20496
-rw------- 1 test1 other 10485760 Sep 26 10:13 aa
$ exit
7. Check the audit file(s)
# cd /var/audit
# ls -l
total 128
-rw------- 1 root root 64370 Sep 26 10:16 20100926081119.20100926081640.sol01
8. Browse the content of this audit file
# praudit 20100926081119.20100926081640.sol01
...
header,134,2,execve(2),,sol01,2010-09-26 10:13:35.371 +02:00
path,/usr/sbin/mkfile
attribute,100555,root,bin,3,853,0
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
header,102,2,open(2) - read,fe,sol01,2010-09-26 10:13:35.371 +02:00
path,/var/ld/ld.config
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,failure: No such file or directory,-1
header,114,2,munmap(2),,sol01,2010-09-26 10:13:35.371 +02:00
argument,1,0xfef8e000,addr
argument,2,0x10000,len
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
header,168,2,memcntl(2),,sol01,2010-09-26 10:13:35.371 +02:00
argument,1,0xfee80000,base
argument,2,0x1e4b0,len
argument,3,0x4,cmd
argument,4,0x3,arg
argument,5,0x0,attr
argument,6,0x0,mask
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
header,114,2,munmap(2),,sol01,2010-09-26 10:13:35.372 +02:00
argument,1,0xfefb0000,addr
argument,2,0x8000,len
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
header,139,2,open(2) - read,write,creat,trunc,,sol01,2010-09-26 10:13:35.386 +02:00
path,/export/home/test1/aa
attribute,100600,test1,other,7,5,0
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,3
header,150,2,close(2),,sol01,2010-09-26 10:13:35.429 +02:00
argument,1,0x3,fd
path,/export/home/test1/aa
attribute,100600,test1,other,7,5,0
subject,test1,test1,other,test1,other,704,3761557250,9027 71168 10.1.0.1
return,success,0
...
As you can see clear here is executed command mkfile and created file, named aa in to the home directory of the user. Of course in above example I use only few of the features in the audit subsystem of Solaris, but in next article will try to present more in deep the options
Subscribe to:
Posts (Atom)