Jenkins part8

Jenkins part8

Class 28th Jenkins May 20th

Prometheus and Grafana 

Prometheus and Grafana are two widely used tools for monitoring and Observability in cloud-native environments 

Prometheus main query based ( monitoring cpu memory,network bits it get the data from nodes and transferred to Grafana there we can see visually,graphic format)

Prometheus and Grafana both are open source monitoring and alerting toolkit originally developed by  soundcloud  and is now part of the cloud native computing foundation(CNCF)

Features:

Time-series database: Stores metrics as time-series data.

Pull-based metrics collection:Scrapes metrics from targets (applications, services )over HTTP

Powerful query language (PromQL):Allows flexible querying and data analysis.

Alerting:Integrated with Alert manager to send alerts via email,slack,pagerduty,etc.

Service discovery:Supports Kubernetes,Ec2,consul,and static Configurations.

Cloud Watch( This also monitor the health check of the server)
Cloud trail (This monitor complete account)
Prometheus default port 9090 and Grafana default port  3000 ,Prometheus collect data source of the server information Grafana with that data showing in Graph GUI 
Grafana : Grafana Open source data visualization and monitoring tool used to display metrics from Prometheus and other data sources.
Features : Supports multiple data source:works with Prometheus,InfluxDB,Mysql ,AWS Cloud Watch ,Elasticsearch ,etc.
Rich Visualization Graphs,heatmaps,pie charts and dashboard 
Alerting system :Triggers alerts when thresholds are breached.

Practical

Step1: Need to create the servers 3 one Prometheus ,2 for create worker1,worker2 , for understand two enough i have not create worker3

Created Three servers 

Server1 

[root@ip-10-0-2-7 ~]# hostnamectl set-hostname  prometgrafa 

[root@ip-10-0-2-7 ~]# exit

logout

[ec2-user@ip-10-0-2-7~]$ sudo -i

[root@prometgrafa ~]#

Server2

[ec2-user@ip-10-0-1-114 ~]$ sudo -i

[root@ip-10-0-1-114 ~]# hostnamectl set-hostname worker1

[root@ip-10-0-1-114 ~]# exit

logout

[ec2-user@ip-10-0-1-114 ~]$ sudo -i

Server3

[ec2-user@ip-10-0-0-39 ~]$ sudo -i

[root@ip-10-0-0-39 ~]# hostnamectl set-hostname worker2

[root@ip-10-0-0-39 ~]# exit

logout

[root@worker2~]#

Step2: Copy past prometgrafa server and execute 

https://github.com/cloudcomputingintelugu/ccit-scripts/blob/main/promgraf.sh

[ec2-user@ip-10-0-2-7 ~]$ sudo -i

[root@worker2 ~]# vi pandg.sh

[root@worker2 ~]# sh pandg.sh

Executed Successfully

Step3: for the script promgraf.sh taken from #NODEEXPORTER line 79 upto end copy 

run in worker1 and worker2

[root@worker1 ~]# vi node.sh

[root@worker1 ~]# sh node.sh

[root@worker2 ~]# vi node.sh

[root@worker2 ~]# sh node.sh

Executed Successfully

Step4: Need to check  Prometheus  and Grafana server how to configure using public ip

Select the status go to target: first endpoint state automatically up because it is same server,remaing to down due to it is different servers need to configure the node

Step5:As given script prometgrafa server below path need edit the hosts file add public ip of the two worker servers and save 
example script:
#3.101.56.72  worker-1
#54.193.223.22 worker-2

#sudo vim /etc/hosts

[root@prometgrafa ~]# sudo vim /etc/hosts
[root@prometgrafa ~]# cat /etc/hosts
127.0.0.1   localhost localhost.localdomain localhost4 localhost4.localdomain4
::1         localhost6 localhost6.localdomain6
52.212.144.119  worker-1
108.129.132.47  worker-2

After refresh ,worker1,worker2 also went up now.

See here below inbuilt some query there you can select cpu take any and execute ,you see below ouput

get the raw data from three servers ,If you more GUI dashboard level you need send the data 

better view in Grafana.

cpu

 Memory

Above Grafana also included ,if you are facing issue ,Please use below command execute the server 

[root@prometgrafa ~]#wget https://dl.grafana.com/oss/release/grafana-10.3.3-1.x86_64.rpm

[root@prometgrafa ~]#sudo yum install grafana-10.3.3-1.x86_64.rpm -y

[root@prometgrafa ~]#ssudo systemctl daemon-reexec

[root@prometgrafa ~]#ssudo systemctl start grafana-server

[root@prometgrafa ~]#ssudo systemctl enable grafana-server

[root@prometgrafa ~]#ssudo systemctl status grafana-server

Grafana default port 3000, default username and password admin/ admin

Step6: Click add new dashboard add visualization ,need click configure a new data source 

our Source is Prometheus ,See below we have many data source options 

You need give your Prometheus url and click save & test

Step7: Now  configure this to dashboard  if you have knowledge create your own dashboard ,elase we can use some existing templates 1860 import dashboard click load and select you data source prometheus  and click import

 This prometheus  local exist server GUI

 This for Work1 healthchecks

  

--Thanks 

                                              Communicate to Post Build status to slack 

Post Build actions in Jenkins communicate to slack Messager channel

Step1: Launch one instance created

Step2: Create one slack account sign-in with Gmail account Click Create workspace 

Select start with free

https://slack.com/

Give Workspace name and username click next ,if you want add you team members also we can add

as of now not added

Step3: Create new Channel >Blank Channel > radio button Public  give channel name

#ccitbuildstatus

Now Slack channel ready

Step4: Which created Instance install jenkins git and java in the server 

[ec2-user@ip-10-0-3-187 ~]$ sudo -i

[root@ip-10-0-3-187 ~]# yum install -y java-17-amazon-corretto-devel git

[root@ip-10-0-3-187 ~]#sudo wget -O /etc/yum.repos.d/jenkins.repo \
https://pkg.jenkins.io/redhat-stable/jenkins.repo
[root@ip-10-0-3-187 ~]#
sudo rpm --import https://pkg.jenkins.io/redhat-stable/jenkins.io-2023.key
[root@ip-10-0-3-187 ~]# sudo yum install jenkins
click y

[root@ip-10-0-3-187  ~]# systemctl status jenkins
[root@ip-10-0-3-187  ~]# systemctl start jenkins
[root@ip-10-0-3-187  ~]# cd /var/lib/jenkins

[root@ip-10-0-3-187 jenkins]# ls -lrt

total 20

drwxr-xr-x 3 jenkins jenkins   21 May 22 09:16 %C

-rw-r--r-- 1 jenkins jenkins   64 May 22 09:16 secret.key

-rw-r--r-- 1 jenkins jenkins    0 May 22 09:16 secret.key.not-so-secret

drwxr-xr-x 2 jenkins jenkins    6 May 22 09:16 plugins

drwxr-xr-x 2 jenkins jenkins    6 May 22 09:16 jobs

-rw-r--r-- 1 jenkins jenkins  156 May 22 09:16 hudson.model.UpdateCenter.xml

-rw-r--r-- 1 jenkins jenkins  171 May 22 09:16 jenkins.telemetry.Correlator.xml

drwxr-xr-x 2 jenkins jenkins   24 May 22 09:16 userContent

-rw-r--r-- 1 jenkins jenkins 1037 May 22 09:16 nodeMonitors.xml

drwxr-xr-x 3 jenkins jenkins   56 May 22 09:16 users

drwx------ 2 jenkins jenkins   91 May 22 09:16 secrets

-rw-r--r-- 1 jenkins jenkins 1660 May 22 09:16 config.xml

drwxr-xr-x 2 jenkins jenkins   67 May 22 09:16 updates

[root@ip-10-0-3-187  ~]# cd /var/lib/jenkins/secrets/

[root@ip-10-0-3-187  secrets]# cat initialAdminPassword

Step4: http://3.249.93.238:8080/ login with initialAdminPassword password give admin password 

create job and build 

Step5:After build Jenkin folder 

[root@ip-10-0-3-187 secrets]# cd /var/lib/jenkins/

[root@ip-10-0-3-187 jenkins]# ls -lrt

total 52

drwxr-xr-x  3 jenkins jenkins   21 May 22 09:16 %C

-rw-r--r--  1 jenkins jenkins   64 May 22 09:16 secret.key

-rw-r--r--  1 jenkins jenkins    0 May 22 09:16 secret.key.not-so-secret

-rw-r--r--  1 jenkins jenkins  156 May 22 09:16 hudson.model.UpdateCenter.xml

-rw-r--r--  1 jenkins jenkins  171 May 22 09:16 jenkins.telemetry.Correlator.xml

drwxr-xr-x  2 jenkins jenkins   24 May 22 09:16 userContent

-rw-r--r--  1 jenkins jenkins 1037 May 22 09:16 nodeMonitors.xml

-rw-r--r--  1 jenkins jenkins 1660 May 22 09:16 config.xml

drwxr-xr-x 90 jenkins jenkins 8192 May 22 09:36 plugins

-rw-------  1 jenkins jenkins 1680 May 22 09:36 identity.key.enc

drwxr-xr-x  2 jenkins jenkins  149 May 22 09:36 updates

-rw-r--r--  1 jenkins jenkins  370 May 22 09:36 hudson.plugins.git.GitTool.xml

drwxr-xr-x  2 jenkins jenkins   32 May 22 09:36 logs

drwxr-xr-x  3 jenkins jenkins   56 May 22 09:36 users

-rw-r--r--  1 jenkins jenkins  182 May 22 09:37 jenkins.model.JenkinsLocationConfiguration.xml

-rw-r--r--  1 jenkins jenkins    7 May 22 09:37 jenkins.install.InstallUtil.lastExecVersion

-rw-r--r--  1 jenkins jenkins    7 May 22 09:37 jenkins.install.UpgradeWizard.state

drwxr-xr-x  3 jenkins jenkins   22 May 22 09:37 jobs

drwxr-xr-x  3 jenkins jenkins   22 May 22 09:37 workspace

drwx------  2 jenkins jenkins  237 May 22 09:37 secrets

Step6: We need install the plugin in Jenkins 

Dashboard >manage Jenkins >Plugins >Available Plugins  >type slack 

Choose Slack Notification  click install After installed successfully.

Step7:Need to configure slack 

 Manage Jenkins >System 

 here we need give workspace credential and channel information need to add

Prior to that We need configure Jenkins in slack 

Step8:Type Jenkins CI and the Click add to Slack 

Step9: add which channel to communicate 

Step10: After click Add Jenkins CI Integration ,they will give instructions please do follow

Step11: Team Sub domain name take and configured in jenkins system configuration

 ccit-epl5538  

Click Add credentials  take the Integration token credential ID iLba4X9G2sX2T3AZYPgeLcPR

Step12:
After added secret text select credentials secret text  and then click save ,With out that build will alert will not go so need remember this 

Step13:

Create one New job 

>New item >  name : ccitpipelinejob  >Select Pipleline option  click Ok 

pipeline {

    agent any

    stages {

        stage('Code') {

            steps {

                echo 'This is Code stage'

            }

        }

        stage('Dev') {

            steps {

                echo 'This is Dev stage'

            }

        }

           stage('Test') {

            steps {

                echo 'This is Test stage'

            }

        }

           stage('Deploy') {

            steps {

                echo 'This is Deplyoment stage'

            }

        }

    }

}

Build Success All stage below, but we need setup Post build setup for slack alert

Step14:

Here below click the job and select the option pipline script syntax , given your slack channel name ccitbuildstatus and Click Generate pipeline script, copy generate the text 

Step15:After copy the above text your job pipline script add below script and save and Build click the Job 

pipeline {

    agent any

    stages {

        stage('Code') {

            steps {

                echo 'This is Code stage'

            }

        }

        stage('Dev') {

            steps {

                echo 'This is Dev stage'

            }

        }

           stage('Test') {

            steps {

                echo 'This is Test stage'

            }

        }

           stage('Deploy') {

            steps {

                echo 'This is Deplyoment stage'

            }

        }

       }    

       post {

            success{

              slackSend channel: 'ccitbuildstatus', message: 'Build Success !...', tokenCredentialId: '3d9bccaf-39e3-4455-b1a4-c4e6f620da31'                }

             failure{

               slackSend channel: 'ccitbuildstatus', message: 'Build Failed !...', tokenCredentialId: '3d9bccaf-39e3-4455-b1a4-c4e6f620da31'                }

        }

    

}

Step16:See Alert capture Build success , we will plan to do one some wrong changes make build failed ,will see build failed alert received or not

Build Failed 

Stage code script remove o   ech 'This is Code stage' , so it was failed in code stage alert also received in the slacked Failed Build

 --Thanks