Beyond the limits - environment (cont 2)

Comment from Paul for the article make me rerun the tests in different conditions: I update the memory of both machines to 1024 MB and use scripts (see below)

[root@centos ~]# cat z1.sh
TEMPVAR=A
for i in {1..1000};
        do
        TEMPVAR=${TEMPVAR}$TEMPVAR;
        a=$(echo 2^$i|bc)
        b=${#TEMPVAR}
        if [ $a -ne $b ]
        then echo KO - $a, $b
        else echo $a
        fi
done
[root@centos ~]# cat z1.1.sh
TEMPVAR=A
for i in {1..1000};
        do
        TEMPVAR=${TEMPVAR}$TEMPVAR;
        a=$(echo 2^$i+1|bc)
        b=$(echo $TEMPVAR|wc -c)
        if [ $a -ne $b ]
        then echo KO - $a, $b
        else echo $a
        fi
done

Solaris
On Solaris independently of the amout of memory and the way of get length of environment variable the result is same:

bash-3.00# vmstat 1 2
 kthr      memory            page            disk          faults      cpu
 r b w   swap  free  re  mf pi po fr de sr cd f0 s0 --   in   sy   cs us sy id
 0 0 0 1075080 616428 152 3085 0 0 0  0 204 10 -0 3  0  309 3345 1511 14 19 67
 1 0 0 1407772 753456 18 50  0  0  0  0  0  0  0  0  0  304  324  142  0  1 99

It seems 32MB is internal limitation in bash (or OS) in Solaris

bash-3.00# ./z1.sh
2
4
<snip>
8388608
16777216
33554432
./z1.sh: fork: Not enough space
./z1.sh: line 7: [: -ne: unary operator expected
./z1.sh: fork: Not enough space
bash-3.00# ./z1.1.sh
3
5
<snip>
16777217
33554433
./z1.1.sh: fork: Not enough space
./z1.1.sh: fork: Not enough space
KO - ,
./z1.1.sh: fork: Not enough space

And the execution times follow the common sense

bash-3.00# time ./z1.sh
2
4
<snip>
16777216
33554432
real    0m9.670s
user    0m6.454s
sys     0m3.014s
bash-3.00# time ./z1.1.sh
3
5
<snip>
16777217
33554433
real    0m13.709s
user    0m8.040s
sys     0m5.352s


Linux
But on linux

[root@centos ~]# vmstat 1 2
procs -----------memory---------- ---swap-- -----io---- --system-- -----cpu------
 r  b   swpd   free   buff  cache   si   so    bi    bo   in   cs us sy id wa st
 0  0  10416 1016704    252   4072   75  804   318   828 1008   77 19 44 32  4  0
 0  0  10416 1016704    252   4096    0    0     0     0 1015   14  0  1 99  0  0


my method of count permit me to reach bigger length

[root@centos ~]# ./z1.sh
2
4
<snip>
67108864
134217728
./z1.sh: xmalloc: cannot allocate 536870913 bytes (0 bytes allocated)
[root@centos ~]# ./z1.1.sh
3
5
<snip>
134217729
268435457
./z1.1.sh: xmalloc: cannot allocate 1073741825 bytes (0 bytes allocated)
KO - 536870913, 0
./z1.1.sh: xmalloc: cannot allocate 1073741825 bytes (0 bytes allocated)
 

and need less execution time (with limit to 2^26)

[root@centos ~]# time ./z1.sh
2
4
<snip>
33554432
67108864
real    1m58.730s
user    0m33.084s
sys     1m27.562s
[root@centos ~]# time ./z1.1.sh
3
5
<snip>
33554433
67108865
real    1m36.457s
user    0m50.206s
sys     0m47.815s



Final conclusion: this need further investigations :)

Oracle DB for system administrators, part9 (shell scripts) cont.

I decide to continue the serie with some ideas about the security. For the test purposes security is not so important, but for production and related (preproduction for example) this is considerable point.
On the previous post i use just plain command line and all the valued parameters as username and password are visible in command line:
[oracle@rh-or ~]$ sqlplus -S romeo/pass1234@orcl @date.sql "Today is "

So everyone who have access to the server can see them. If I try to use other technique to try to hide them with environment variable like this:
[oracle@rh-or ~]$ O_USER=romeo
[oracle@rh-or ~]$ O_PASS=pass1234
[oracle@rh-or ~]$ sqlplus -S $O_USER/$O_PASS@orcl @date.sql "Today is "

this will hide information from command line, but it is still visible via environment parameters of the process. For example in Solaris command pargs will provide such information
(https://unixswing.blogspot.com/2018/09/solaris-commands-for-process-management_22.html)
Lets rewrite the script with some inline code
sqlplus -S /nolog << EOD
connect romeo/pass1234@orcl
@date.sql
exit
EOD

The above script will expose in process list only fact of run sqlplus and nothing more. The rest of the information will be "entered" like in interactive session w/o any traces for other users.
Be aware that in the last line of code (word EOD) you should have no other symbols except mentioned word. Otherwise the script will not work on expected manner.
The rights of such script should be set to 700 to avoid disclosure of sensitive information to the extraneous. The only issue is you cant hide the information in script from the user will run it because in UNIX/Linux shells if you want to execute script you should have right to read it.

Dealing with variables in shell environment - canonical way

One of the very important elements of interactive work with UNIX/Linux is a shell, especially variables in shell environment. To work with variables sometime can be easy, sometime hard. So here I will try to show some of the ideas how to deal with them.

1. how to see the variable: use one of the commands:

set|grep VAR
or

echo $VAR

2. how to set variable:

VAR=value;export VAR

or if you want to add some value to the existing in the variable

VAR=$VAR:value;export VAR

Of course export is need only in case you want to populate value to subinstrances of current shell. I use this way because it work on bourne shell, korn shell and bash. If you are sure script will work only in bash and/or korn shell use short form:

export VAR=value

3. how to destroy the variable:

unset VAR

I can use

VAR="";export VAR

but this only will "zero" the content and not destroy the variable

4. how to change (inside) the content of variable. You can make it entering the entire string (or number), but this little script will help you do it on the fly. Do not forget script is just an example and you should "tune" it to do the real work

VAR=`echo $VAR|sed s/string//g`;export VAR

Shell in nutshell or little tips and tricks

Today I will try to show  some tricks in UNIX/Linux shells maybe you do not know:

- use . (dot) to execute command or script in the current instance of the shell, this help much when you try to set some variables or change to directory with long and complicated path

- use - (dash) to go to previous working directory. this do not work in bourne shell

- use ${n..nn} in bash to generate sequence of numbers. Maybe was something similar in korn shell, but I can't remember.

$ printf "%03d " {1..100}

Also you can use seq program (mainly available in Linux)

$ printf "%03d " `seq 1 100`

- use bc to make arithmetic calculations on the fly:

# echo 2^15|bc

32768

- printf is very helpfull when you try to create well formatted output. Just follow the same format as in C

To be continued (maybe)...