/proc: limits, cwd, filedescriptors, environment variables

pseudo-file system /proc provides access to important process details that will help in running process debugging.

It’s nice documented and a lot of info can be found in man proc

Let’s highlight some useful details

In order to find pid of running process ps, pgrep will help on that

root@ip-172-31-30-213:/# pgrep -fl mysql
809 /usr/sbin/mysqld

Runtime limits on existing process can be found in /proc/[pid]/limits

root@ip-172-31-30-213:/# cat /proc/809/limits
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 13412 13412 processes
Max open files 1024 4096 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 13412 13412 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us

usecase: when process reach its limits, and configuration limits will vary from what was during process start.

/proc/[pid]/fd/ – list of open file descriptors and links to files

root@ip-172-31-30-213:/# ls /proc/809/fd
0 11 14 17 2 22 25 28 30 33 36 39 41 44 47 5 52 55 58 60 63 66 69 71 74 77 8 82 9
1 12 15 18 20 23 26 29 31 34 37 4 42 45 48 50 53 56 59 61 64 67 7 72 75 78 80 83
10 13 16 19 21 24 27 3 32 35 38 40 43 46 49 51 54 57 6 62 65 68 70 73 76 79 81 86

usecase: find list of open files that process read/write. In conjunction with strace helps to identify what exactly being written/read to file.

/proc/809/environ – details on runtime environment variable

root@ip-172-31-30-213:/# cat /proc/809/environ |sed ‘s/\x0/\n/g’
PATH=/usr/local/sbin:/usr/local/bin:/usr/bin:/usr/sbin:/sbin:/bin
TERM=linux
HOME=/etc/mysql
RUNLEVEL=2
PREVLEVEL=N
UPSTART_EVENTS=runlevel
UPSTART_JOB=mysql
UPSTART_INSTANCE=

usecase: many unexpected behavior caused by wrong env variable setting.

/proc/[pid]/cwd show current workdirectory of running process

root@ip-172-31-30-213:/# pgrep -fl python
8419 ./python -c import time; time.sleep(180)
root@ip-172-31-30-213:/# ls -l /proc/8419/cwd
lrwxrwxrwx 1 root root 0 Oct 2 21:50 /proc/8419/cwd -> /usr/bin

usecase: when multiple tools with same name but different version available on server (java, python, apache) and you have to figure out which version exactly running now.

Log everything you do in Putty

Haven’t you fall into situation when you removed some lines in file without making backup. Or you need to recall what you did on specific host at specific day.

Putty supports nice features for recording all your activities.

Got to Session, select desired saved session (or Default Settings) and press Load

putty_session

Select ‘All session output’ and set Log file name to ‘C:\logs\&Y_&M_&D_&H.log’

&Y – year

&M – month

&D – day

&H – hour

Back to Session tab, select session and press ‘Save’

After that changes all session that uses your settings will be recorded to file.

Top plugins which have to be installed before start of work

1. Timestamper

Added timestamp in logs. To add timestamp to logs do 2 steps:

– Go to Jenkins -> Manage Jenkins -> System configurations ->

This format will add the following timestamp to logs

18/09 09:18:42      [copy] Copying 1 file to D:\Hybris5.2\hybris\bin\platform\ext\impex\resources
18/09 09:18:43      [copy] Copying 1 file to D:\Hybris5.2\hybris\bin\platform\ext\validation\resources

– In each jobs configuration in Build Environment field add mark near Add timestamps to the Console Output


2.Copy Artifact Plugin

 In case you need to share artifacts between your jobs this is a good plugin to use. Also it can be useful if you want to share some environment variables, below I will show how to do it.
Artifact is some result of job, however if this result not saved as file (txt, jar, war etc.) you wont be able to share it. For instance, your jobs execute shell commands but it not saved in output file, so the re is no artifact and wouldn’t be able to share it.
So that lets save in file hello.txt environment variables of current job. In my case job runs on Windows machine so I use Windows batch command in Build part of job and save variables like BUILD_URL, COMPUTERNAME.
In Post-build Actions part add Archive the artifacts and put the name or pattern e.g. *.txt
Then you can use the shares artifacts in another jobs. To do this add in Build  part one mere step Copy artifacts from another project there you can put which job to use and what artifact to copy.

In way example I want to get the environment variables so I need to inject them from hello.txt file. For this need I use one more plugin Environment Injector Plugin  see next.

3. Environment Injector Plugin

To inject environment variables add step Inject Environment variables


4. Email Extension Plugin

In Post-build Actions usually exist step as E-mail Notification which will send an email when jobs will finished. Sometimes more details are needed not only that jobs has been finished. It could be git revision, build date, build URL, environment and other. For this purpose can be used Email Extension Plugin. Add Editable Email Notification  in Post-build Actions.  And configure the body of email using environment variables of jobs.
</STYLE>
<BODY>
<h5 style=”color: #1F497D; font-family: Calibri”>Hi Team</h5>
<p></p>
<h3 style=”color: #212121; font-family: Calibri”>R1</h3>
<p>QA-R1 is up</p>
<p style=”color: #212121; font-family:Courier New; font-size: 10px”>JENKINS_BUILD_NUM=${BUILD_NUMBER}</p>
<p>*******************************</p>
<p>test_in_number = ${TRIGGERED_BUILD_NUMBERS_test_in}</p>
<p>tests_in_result = ${BUILT_ON}</p>
best regards,
</BODY>

In system configuration add:

5. Build Flow Plugin

It is highly recommended to use this plugin if your jobs is quite complicated. There could be some conditions steps and and as more you have them as harder to somebody to understand hows your job is working. Due to this use easy DSL language also it is possible to write flow with groovy.
!!! To create such jobs click New Item and choose Build Flow. Then you will get not Build part but Flow.

6. Artifactory Plugin

After installation add to Jenkins -> Configurations  the configuration for artifactory


In the job body add in Artifactory Configuration – > Artifactory Server and target repositoryIn Published Artifacts add where to take and to put the artifacts

7.LogParser Plugin

To easy and quick check logs if it contains any errors or severe use this plugin. Add to system configuration Console Output Parsing part:
Log Parser file containe the rules:
 ok /platform/
ok /next nodes/
ok /No accessor found for attribute ‘navigationNodeList’/
ok /does not belong to known type. Maybe you have added a new type without updating your system/# match line starting with ‘error ‘, case-insensitive
error / ERROR/

# list of warnings here…
warning /WARN/

# create a quick access link to lines in the report containing ‘INFO’
#info /INFO/

# each line containing ‘BUILD’ represents the start of a section for grouping errors and warnings found after the line.
# also creates a quick access link.
start /Launching a JVM…

In Jenkins job add to the Post-build actions -> Console potput (build log) parsing and choose the file with pattern

 

8. Parameterized Trigger plugin

 

9. Mailer Plugin

10. Job Configuration History Plugin

11. Groovy

12. Git Plugin

13. Extended Choice Parameter Plugin

14. Build Monitor View

15. Ant Plugin

In system configuration tab

 16.Slicing Plugin

To configure multiple jobs at the same time

 17. Groovy script console

To configure multiple jobs at the same time and run script on groovy

16. Naginator Plugin

Use this plugin to retry run job if it fail. Add in Post-build Actions -> Retry build after failure