Comment from Paul for the article make me rerun the tests in different conditions: I update the memory of both machines to 1024 MB and use scripts (see below)
[root@centos ~]# cat z1.sh
TEMPVAR=A
for i in {1..1000};
do
TEMPVAR=${TEMPVAR}$TEMPVAR;
a=$(echo 2^$i|bc)
b=${#TEMPVAR}
if [ $a -ne $b ]
then echo KO - $a, $b
else echo $a
fi
done
[root@centos ~]# cat z1.1.sh
TEMPVAR=A
for i in {1..1000};
do
TEMPVAR=${TEMPVAR}$TEMPVAR;
a=$(echo 2^$i+1|bc)
b=$(echo $TEMPVAR|wc -c)
if [ $a -ne $b ]
then echo KO - $a, $b
else echo $a
fi
done
Solaris
On Solaris independently of the amout of memory and the way of get length of environment variable the result is same:
bash-3.00# vmstat 1 2
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr cd f0 s0 -- in sy cs us sy id
0 0 0 1075080 616428 152 3085 0 0 0 0 204 10 -0 3 0 309 3345 1511 14 19 67
1 0 0 1407772 753456 18 50 0 0 0 0 0 0 0 0 0 304 324 142 0 1 99
It seems 32MB is internal limitation in bash (or OS) in Solaris
bash-3.00# ./z1.sh
2
4
<snip>
8388608
16777216
33554432
./z1.sh: fork: Not enough space
./z1.sh: line 7: [: -ne: unary operator expected
./z1.sh: fork: Not enough space
bash-3.00# ./z1.1.sh
3
5
<snip>
16777217
33554433
./z1.1.sh: fork: Not enough space
./z1.1.sh: fork: Not enough space
KO - ,
./z1.1.sh: fork: Not enough space
And the execution times follow the common sense
bash-3.00# time ./z1.sh
2
4
<snip>
16777216
33554432
real 0m9.670s
user 0m6.454s
sys 0m3.014s
bash-3.00# time ./z1.1.sh
3
5
<snip>
16777217
33554433
real 0m13.709s
user 0m8.040s
sys 0m5.352s
Linux
But on linux
[root@centos ~]# vmstat 1 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 10416 1016704 252 4072 75 804 318 828 1008 77 19 44 32 4 0
0 0 10416 1016704 252 4096 0 0 0 0 1015 14 0 1 99 0 0
my method of count permit me to reach bigger length
[root@centos ~]# ./z1.sh
2
4
<snip>
67108864
134217728
./z1.sh: xmalloc: cannot allocate 536870913 bytes (0 bytes allocated)
[root@centos ~]# ./z1.1.sh
3
5
<snip>
134217729
268435457
./z1.1.sh: xmalloc: cannot allocate 1073741825 bytes (0 bytes allocated)
KO - 536870913, 0
./z1.1.sh: xmalloc: cannot allocate 1073741825 bytes (0 bytes allocated)
and need less execution time (with limit to 2^26)
[root@centos ~]# time ./z1.sh
2
4
<snip>
33554432
67108864
real 1m58.730s
user 0m33.084s
sys 1m27.562s
[root@centos ~]# time ./z1.1.sh
3
5
<snip>
33554433
67108865
real 1m36.457s
user 0m50.206s
sys 0m47.815s
Final conclusion: this need further investigations :)
Showing posts with label shell. Show all posts
Showing posts with label shell. Show all posts
Beyond the limits - environment (cont)
4. Accessable command line argument
Now I will try to see how many are acceptable command line arguments from system utils like pgrep, pkill and similar. For this puspose I will use dummy C program which do nothing
#include <stdio.h>
int main(int argc, char *argv[])
{
while(1){};
}
And will generate sequence of random numbers to be fure will have ho false results
for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|md5sum|awk '{printf $1" "}'>>random_arg;done
[root@centos ~]# cat random_arg
d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09
<snip>
1187d29932b7d5827080684f16205330 92398e4719194f13a1a79064c5a69d05 e1cd425a036f77cc6fb6f3002d00bada a0c6d3b48c5e8146b2fe0b5f74383b9c
Let try with shorter arguments
4.1. Linux
Compile the program
[root@centos ~]# cc z4.c
And execute it
[root@centos ~]# ./a.out `cat random_arg `&
[1] 5094
Lets check what I can see with ps
[[root@centos ~]# ps -efl|grep a.out
0 R root 5094 2174 99 85 0 - 403 - 06:13 pts/0 00:00:35 ./a.out d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09 73e522501878712898d1c47342b9bd66
<snip>
f566402f18f3ebe15453d3a74fc1c534 f87fe9ab1f5f85238ce01ca5d62d57e1 d05d9d3b9cb608d8a177434a4cda5c69 43e80e6069a763a0da17abdba8a06b30 d79a21984d368e4b574e4690a749c7af f5c9eddc01dd1a98bdb7ecbd5eba87a7 249a93f99bb66ff2361f722d1de2b
0 R root 5101 2174 0 78 0 - 1001 - 06:13 pts/0 00:00:00 grep a.out
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
and exec
[root@centos ~]# ./z3.sh
OK 1
OK 2
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
So I can see only 123 arguments. Hm, this a big number, but no so much
Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
The same result. So i can see beyond parameter number 123. And sample chech if this depend of number of variables in environment
[root@centos ~]# env|wc -l
20
[root@centos ~]# export BB=1
[root@centos ~]# env|wc -l
21
[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
It seems do not depend and this is internal limitation
4.2. Solaris
Compile the program
bash-3.00# /usr/sfw/bin/gcc z3.c
and generate random numbers
for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|digest -a md5|awk '{printf $1" "}'>>random_arg;done
bash-3.00# cat random_arg
40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b06061f1e4a76e3ced152855f7f5047 89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282 637ff2c7ad50c5f2018f7a865d646667 35bcc0568f9b8b8902fdc4f73fbadb49 34ce610edd36d834c08115de0c8e4568
<snip>
And execute it
bash-3.00# ./a.out `cat random_arg `&
[1] 1536
Lets check what I can see with ps
bash-3.00# ps -efl|grep a.out
0 S root 1539 749 0 40 20 ? 347 ? 07:40:11 pts/1 0:00 grep a.out
0 R root 1536 749 70 89 20 ? 336 07:39:46 pts/1 0:26 ./a.out 40e4b176f31e662ae95ac93e995
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
and exec
bash-3.00# ./z3.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
As you can see Solaris show me only two arguments with SYSV ps, but let me try with pargs
bash-3.00# pargs 1536
1536: ./a.out 40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b060
argv[0]: ./a.out
argv[1]: 40e4b176f31e662ae95ac93e995cf4a4
argv[2]: f1e1c978a8007aa88bfffd675af2d56b
<snip>
argv[256]: 85fa98d8bf33f2b0507ec9714472d17f
OK, thats good, but I should know the PID :)
Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
Ha, I can see only two arguments, this is too small for me
And sample check if this depend of number of variables in environment
bash-3.00# env|wc -l
14
bash-3.00# export BB=1
bash-3.00# env|wc -l
15
bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
It seems do not depend and this is internal limitation
Let see what will happen with shorter arguments
bash-3.00# ./a.out 111 222 333 444 555 666 777 888 999&
[1] 2346
bash-3.00# cat z4.sh
CNT=1
for i in 111 222 333 444 555 666 777 888 999;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
bash-3.00# ./z4.sh
OK 1
OK 2
OK 3
OK 4
OK 5
OK 6
OK 7
OK 8
OK 9
O, all seems OK, but why i cant reach more deeper in argument list. Let keep this question open :)
Now I will try to see how many are acceptable command line arguments from system utils like pgrep, pkill and similar. For this puspose I will use dummy C program which do nothing
#include <stdio.h>
int main(int argc, char *argv[])
{
while(1){};
}
And will generate sequence of random numbers to be fure will have ho false results
for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|md5sum|awk '{printf $1" "}'>>random_arg;done
[root@centos ~]# cat random_arg
d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09
<snip>
1187d29932b7d5827080684f16205330 92398e4719194f13a1a79064c5a69d05 e1cd425a036f77cc6fb6f3002d00bada a0c6d3b48c5e8146b2fe0b5f74383b9c
Let try with shorter arguments
4.1. Linux
Compile the program
[root@centos ~]# cc z4.c
And execute it
[root@centos ~]# ./a.out `cat random_arg `&
[1] 5094
Lets check what I can see with ps
[[root@centos ~]# ps -efl|grep a.out
0 R root 5094 2174 99 85 0 - 403 - 06:13 pts/0 00:00:35 ./a.out d4a798eea8f9b9b5c9751aa8a785b465 da735316b9d27af062c28a0e3655a1f5 43c45dcd136e53d01662a5f9cfa943d9 fd5d2445ebf54a969cd6dff7e8cb3c09 73e522501878712898d1c47342b9bd66
<snip>
f566402f18f3ebe15453d3a74fc1c534 f87fe9ab1f5f85238ce01ca5d62d57e1 d05d9d3b9cb608d8a177434a4cda5c69 43e80e6069a763a0da17abdba8a06b30 d79a21984d368e4b574e4690a749c7af f5c9eddc01dd1a98bdb7ecbd5eba87a7 249a93f99bb66ff2361f722d1de2b
0 R root 5101 2174 0 78 0 - 1001 - 06:13 pts/0 00:00:00 grep a.out
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
and exec
[root@centos ~]# ./z3.sh
OK 1
OK 2
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
So I can see only 123 arguments. Hm, this a big number, but no so much
Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
The same result. So i can see beyond parameter number 123. And sample chech if this depend of number of variables in environment
[root@centos ~]# env|wc -l
20
[root@centos ~]# export BB=1
[root@centos ~]# env|wc -l
21
[root@centos ~]# ./z4.sh
OK 1
<snip>
OK 123
249a93f99bb66ff2361f722d1de2b8ff
f2793da3d8bbab2c7b972c4ff2dc7176
60bc35ea120dc372a5b14a02acc2182a
<snip>
It seems do not depend and this is internal limitation
4.2. Solaris
Compile the program
bash-3.00# /usr/sfw/bin/gcc z3.c
and generate random numbers
for i in {1..256}; do dd if=/dev/urandom bs=1 count=64|digest -a md5|awk '{printf $1" "}'>>random_arg;done
bash-3.00# cat random_arg
40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b06061f1e4a76e3ced152855f7f5047 89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282 637ff2c7ad50c5f2018f7a865d646667 35bcc0568f9b8b8902fdc4f73fbadb49 34ce610edd36d834c08115de0c8e4568
<snip>
And execute it
bash-3.00# ./a.out `cat random_arg `&
[1] 1536
Lets check what I can see with ps
bash-3.00# ps -efl|grep a.out
0 S root 1539 749 0 40 20 ? 347 ? 07:40:11 pts/1 0:00 grep a.out
0 R root 1536 749 70 89 20 ? 336 07:39:46 pts/1 0:26 ./a.out 40e4b176f31e662ae95ac93e995
OK, but let see if i can find all the arguments with ps:
CNT=1
for i in `cat random_arg `;
do
if [ `ps -efl|grep $i|grep -v grep|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
and exec
bash-3.00# ./z3.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
As you can see Solaris show me only two arguments with SYSV ps, but let me try with pargs
bash-3.00# pargs 1536
1536: ./a.out 40e4b176f31e662ae95ac93e995cf4a4 f1e1c978a8007aa88bfffd675af2d56b 6b060
argv[0]: ./a.out
argv[1]: 40e4b176f31e662ae95ac93e995cf4a4
argv[2]: f1e1c978a8007aa88bfffd675af2d56b
<snip>
argv[256]: 85fa98d8bf33f2b0507ec9714472d17f
OK, thats good, but I should know the PID :)
Lets try with pgrep
CNT=1
for i in `cat random_arg `;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
Ha, I can see only two arguments, this is too small for me
And sample check if this depend of number of variables in environment
bash-3.00# env|wc -l
14
bash-3.00# export BB=1
bash-3.00# env|wc -l
15
bash-3.00# ./z4.sh
OK 1
OK 2
6b06061f1e4a76e3ced152855f7f5047
89e3f7ee6744078726a58ba3fe85092a
d1659b7259bf383888e93caee7e26282
<snip>
It seems do not depend and this is internal limitation
Let see what will happen with shorter arguments
bash-3.00# ./a.out 111 222 333 444 555 666 777 888 999&
[1] 2346
bash-3.00# cat z4.sh
CNT=1
for i in 111 222 333 444 555 666 777 888 999;
do
if [ `pgrep -f $i|wc -l` -eq 1 ]
then echo OK $CNT
else echo $i
fi
CNT=`echo $CNT + 1|bc`
done
bash-3.00# ./z4.sh
OK 1
OK 2
OK 3
OK 4
OK 5
OK 6
OK 7
OK 8
OK 9
O, all seems OK, but why i cant reach more deeper in argument list. Let keep this question open :)
Beyond the limits - environment
1. Prerequisites
My environmnet is based on Solaris 10 10 9/10 with 640 MB of memory and CentOS 5.5 2.6.18-194.26.1.el5 with 512 MB, standard installation both of them run in VMWare environmnet. For shell I use bash (as common)
bash-3.00# bash --version
GNU bash, version 3.00.16(1)-release (i386-pc-solaris2.10)
Copyright (C) 2004 Free Software Foundation, Inc.
[root@centos ~]# bash --version
GNU bash, version 3.2.25(1)-release (i686-redhat-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.
All the tests will be done with default settings of the OS and environment
2. Test the maximum length variable
the script i will use is sample:
TEMPVAR=A
for i in {1..10000};
do
TEMPVAR=${TEMPVAR}$TEMPVAR;
a=$(echo 2^$i+1|bc)
b=$(echo $TEMPVAR|wc -c)
if [ $a -ne $b ]
then echo KO - $a, $b
else echo $a
fi
done
2.1. And the execution in Linux
[root@centos ~]# ./z1.sh
3
5
<snip>
67108865
134217729
./z1.sh: xmalloc: cannot allocate 268435457 bytes (0 bytes allocated)
KO - 268435457, 0
[1]+ Stopped ./z1.sh
[root@centos ~]# vmstat 1 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 10716 229500 25160 228524 66 139 138 178 1011 151 18 18 61 3 0
0 0 10716 229500 25160 228524 0 0 0 0 1013 15 0 0 100 0 0
As you can see above I get error allocating memory beyond the size of free memory
2.2. Execution in Solaris
bash-3.00# ./z1.sh
3
5
<snip>
8388609
16777217
./z1.sh: fork: Not enough space
./z1.sh: line 7: [: 33554433: unary operator expected
33554433
./z1.sh: fork: Not enough space
bash-3.00# vmstat 1 2
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr cd f0 s0 -- in sy cs us sy id
0 0 0 856808 174100 50 1101 0 1 5 0 66 2 -0 3 0 336 801 308 9 7 84
0 0 0 749832 70472 3 43 0 0 0 0 0 27 0 0 0 326 300 185 1 9 90
The similar is situation in Solaris where i can allocate even less that half of free memory
Of course it is not very wise to use such variables, but who knows?
3. Number of commandline arguments
3.1. Sample C program
Next I will try to check what is the maximum of commandline arguments of one sample C program
#include <stdio.h>
int main(int argc, char *argv[])
{
printf("%d\n", argc);
}
and the shell script I will use is
TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./a.out $TEMPVAR)
echo $a
done
3.1.1 On Linux
[root@centos ~]# cc z2.c
[root@centos ~]# ./a.out 1
2
[root@centos ~]# ./a.out 1 2
3
[root@centos ~]# ./z2.sh
3
5
<snip>
524289
1048577
./z2.sh: line 5: ./a.out: Argument list too long
1m of arguments, that pretty much :)
3.1.2 Solaris
bash-3.00# /usr/sfw/bin/gcc z2.c
bash-3.00# ./a.out 1
2
bash-3.00# ./a.out 1 2
3
bash-3.00# ./z2.sh
3
5
<snip>
65537
131073
./z2.sh: line 5: ./a.out: Arg list too long
"Just" 128k, not so much as Linux, but anyway do someone will use so many for practical puspose?
3.2. Sample shell program
Next I will try to check what is the maximum of commandline arguments of one very sample shell program
echo $#
using this shell script
TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./z2test.sh $TEMPVAR)
echo $a
done
This time the results appear mich slower because of the invocation of subshell.
3.2.1 Linux
After 1 hour of waiting I stop the script, but before this I see in top
29827 root 25 0 47028 41m 476 R 99.9 8.3 2:38.86 bash
3.2.2. Solaris
The same (long waiting) happen in Solaris and after 128k i see in prstat
15100 root 72M 69M run 20 0 0:09:01 96% bash/1
As conclusion 128K is the maximum practicle limit of number command line arguments
My environmnet is based on Solaris 10 10 9/10 with 640 MB of memory and CentOS 5.5 2.6.18-194.26.1.el5 with 512 MB, standard installation both of them run in VMWare environmnet. For shell I use bash (as common)
bash-3.00# bash --version
GNU bash, version 3.00.16(1)-release (i386-pc-solaris2.10)
Copyright (C) 2004 Free Software Foundation, Inc.
[root@centos ~]# bash --version
GNU bash, version 3.2.25(1)-release (i686-redhat-linux-gnu)
Copyright (C) 2005 Free Software Foundation, Inc.
All the tests will be done with default settings of the OS and environment
2. Test the maximum length variable
the script i will use is sample:
TEMPVAR=A
for i in {1..10000};
do
TEMPVAR=${TEMPVAR}$TEMPVAR;
a=$(echo 2^$i+1|bc)
b=$(echo $TEMPVAR|wc -c)
if [ $a -ne $b ]
then echo KO - $a, $b
else echo $a
fi
done
2.1. And the execution in Linux
[root@centos ~]# ./z1.sh
3
5
<snip>
67108865
134217729
./z1.sh: xmalloc: cannot allocate 268435457 bytes (0 bytes allocated)
KO - 268435457, 0
[1]+ Stopped ./z1.sh
[root@centos ~]# vmstat 1 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
r b swpd free buff cache si so bi bo in cs us sy id wa st
0 0 10716 229500 25160 228524 66 139 138 178 1011 151 18 18 61 3 0
0 0 10716 229500 25160 228524 0 0 0 0 1013 15 0 0 100 0 0
As you can see above I get error allocating memory beyond the size of free memory
2.2. Execution in Solaris
bash-3.00# ./z1.sh
3
5
<snip>
8388609
16777217
./z1.sh: fork: Not enough space
./z1.sh: line 7: [: 33554433: unary operator expected
33554433
./z1.sh: fork: Not enough space
bash-3.00# vmstat 1 2
kthr memory page disk faults cpu
r b w swap free re mf pi po fr de sr cd f0 s0 -- in sy cs us sy id
0 0 0 856808 174100 50 1101 0 1 5 0 66 2 -0 3 0 336 801 308 9 7 84
0 0 0 749832 70472 3 43 0 0 0 0 0 27 0 0 0 326 300 185 1 9 90
The similar is situation in Solaris where i can allocate even less that half of free memory
Of course it is not very wise to use such variables, but who knows?
3. Number of commandline arguments
3.1. Sample C program
Next I will try to check what is the maximum of commandline arguments of one sample C program
#include <stdio.h>
int main(int argc, char *argv[])
{
printf("%d\n", argc);
}
and the shell script I will use is
TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./a.out $TEMPVAR)
echo $a
done
3.1.1 On Linux
[root@centos ~]# cc z2.c
[root@centos ~]# ./a.out 1
2
[root@centos ~]# ./a.out 1 2
3
[root@centos ~]# ./z2.sh
3
5
<snip>
524289
1048577
./z2.sh: line 5: ./a.out: Argument list too long
1m of arguments, that pretty much :)
3.1.2 Solaris
bash-3.00# /usr/sfw/bin/gcc z2.c
bash-3.00# ./a.out 1
2
bash-3.00# ./a.out 1 2
3
bash-3.00# ./z2.sh
3
5
<snip>
65537
131073
./z2.sh: line 5: ./a.out: Arg list too long
"Just" 128k, not so much as Linux, but anyway do someone will use so many for practical puspose?
3.2. Sample shell program
Next I will try to check what is the maximum of commandline arguments of one very sample shell program
echo $#
using this shell script
TEMPVAR=A
for i in {1..1000}
do
TEMPVAR="$TEMPVAR $TEMPVAR"
a=$(./z2test.sh $TEMPVAR)
echo $a
done
This time the results appear mich slower because of the invocation of subshell.
3.2.1 Linux
After 1 hour of waiting I stop the script, but before this I see in top
29827 root 25 0 47028 41m 476 R 99.9 8.3 2:38.86 bash
3.2.2. Solaris
The same (long waiting) happen in Solaris and after 128k i see in prstat
15100 root 72M 69M run 20 0 0:09:01 96% bash/1
As conclusion 128K is the maximum practicle limit of number command line arguments
Oracle DB for system administrators, part9 (shell scripts) cont.
I decide to continue the serie with some ideas about the security. For the test purposes security is not so important, but for production and related (preproduction for example) this is considerable point.
On the previous post i use just plain command line and all the valued parameters as username and password are visible in command line:
[oracle@rh-or ~]$ sqlplus -S romeo/pass1234@orcl @date.sql "Today is "
So everyone who have access to the server can see them. If I try to use other technique to try to hide them with environment variable like this:
[oracle@rh-or ~]$ O_USER=romeo
[oracle@rh-or ~]$ O_PASS=pass1234
[oracle@rh-or ~]$ sqlplus -S $O_USER/$O_PASS@orcl @date.sql "Today is "
this will hide information from command line, but it is still visible via environment parameters of the process. For example in Solaris command pargs will provide such information
(https://unixswing.blogspot.com/2018/09/solaris-commands-for-process-management_22.html)
Lets rewrite the script with some inline code
sqlplus -S /nolog << EOD
connect romeo/pass1234@orcl
@date.sql
exit
EOD
The above script will expose in process list only fact of run sqlplus and nothing more. The rest of the information will be "entered" like in interactive session w/o any traces for other users.
Be aware that in the last line of code (word EOD) you should have no other symbols except mentioned word. Otherwise the script will not work on expected manner.
The rights of such script should be set to 700 to avoid disclosure of sensitive information to the extraneous. The only issue is you cant hide the information in script from the user will run it because in UNIX/Linux shells if you want to execute script you should have right to read it.
On the previous post i use just plain command line and all the valued parameters as username and password are visible in command line:
[oracle@rh-or ~]$ sqlplus -S romeo/pass1234@orcl @date.sql "Today is "
So everyone who have access to the server can see them. If I try to use other technique to try to hide them with environment variable like this:
[oracle@rh-or ~]$ O_USER=romeo
[oracle@rh-or ~]$ O_PASS=pass1234
[oracle@rh-or ~]$ sqlplus -S $O_USER/$O_PASS@orcl @date.sql "Today is "
this will hide information from command line, but it is still visible via environment parameters of the process. For example in Solaris command pargs will provide such information
(https://unixswing.blogspot.com/2018/09/solaris-commands-for-process-management_22.html)
Lets rewrite the script with some inline code
sqlplus -S /nolog << EOD
connect romeo/pass1234@orcl
@date.sql
exit
EOD
The above script will expose in process list only fact of run sqlplus and nothing more. The rest of the information will be "entered" like in interactive session w/o any traces for other users.
Be aware that in the last line of code (word EOD) you should have no other symbols except mentioned word. Otherwise the script will not work on expected manner.
The rights of such script should be set to 700 to avoid disclosure of sensitive information to the extraneous. The only issue is you cant hide the information in script from the user will run it because in UNIX/Linux shells if you want to execute script you should have right to read it.
Dealing with variables in shell environment - canonical way
One of the very important elements of interactive work with UNIX/Linux is a shell, especially variables in shell environment. To work with variables sometime can be easy, sometime hard. So here I will try to show some of the ideas how to deal with them.
1. how to see the variable: use one of the commands:
set|grep VAR
or
echo $VAR
2. how to set variable:
VAR=value;export VAR
or if you want to add some value to the existing in the variable
VAR=$VAR:value;export VAR
Of course export is need only in case you want to populate value to subinstrances of current shell. I use this way because it work on bourne shell, korn shell and bash. If you are sure script will work only in bash and/or korn shell use short form:
export VAR=value
3. how to destroy the variable:
unset VAR
I can use
VAR="";export VAR
but this only will "zero" the content and not destroy the variable
4. how to change (inside) the content of variable. You can make it entering the entire string (or number), but this little script will help you do it on the fly. Do not forget script is just an example and you should "tune" it to do the real work
VAR=`echo $VAR|sed s/string//g`;export VAR
1. how to see the variable: use one of the commands:
set|grep VAR
or
echo $VAR
2. how to set variable:
VAR=value;export VAR
or if you want to add some value to the existing in the variable
VAR=$VAR:value;export VAR
Of course export is need only in case you want to populate value to subinstrances of current shell. I use this way because it work on bourne shell, korn shell and bash. If you are sure script will work only in bash and/or korn shell use short form:
export VAR=value
3. how to destroy the variable:
unset VAR
I can use
VAR="";export VAR
but this only will "zero" the content and not destroy the variable
4. how to change (inside) the content of variable. You can make it entering the entire string (or number), but this little script will help you do it on the fly. Do not forget script is just an example and you should "tune" it to do the real work
VAR=`echo $VAR|sed s/string//g`;export VAR
Shell in nutshell or little tips and tricks
Today I will try to show some tricks in UNIX/Linux shells maybe you do not know:
- use . (dot) to execute command or script in the current instance of the shell, this help much when you try to set some variables or change to directory with long and complicated path
- use - (dash) to go to previous working directory. this do not work in bourne shell
- use ${n..nn} in bash to generate sequence of numbers. Maybe was something similar in korn shell, but I can't remember.
$ printf "%03d " {1..100}
Also you can use seq program (mainly available in Linux)
$ printf "%03d " `seq 1 100`
- use bc to make arithmetic calculations on the fly:
# echo 2^15|bc
32768
- printf is very helpfull when you try to create well formatted output. Just follow the same format as in C
To be continued (maybe)...
- use . (dot) to execute command or script in the current instance of the shell, this help much when you try to set some variables or change to directory with long and complicated path
- use - (dash) to go to previous working directory. this do not work in bourne shell
- use ${n..nn} in bash to generate sequence of numbers. Maybe was something similar in korn shell, but I can't remember.
$ printf "%03d " {1..100}
Also you can use seq program (mainly available in Linux)
$ printf "%03d " `seq 1 100`
- use bc to make arithmetic calculations on the fly:
# echo 2^15|bc
32768
- printf is very helpfull when you try to create well formatted output. Just follow the same format as in C
To be continued (maybe)...
Subscribe to:
Posts (Atom)