dpkg

List all installed packages :

1
#dpkg -l

Dump installed packages to a file :

1
# dpkg --get-selections > packages

Reinstall listed packages :

1
# dpkg --set-selections < packages

Search the package that install a file :

1
# dpkg -S /usr/share/doc/linux-image-generic
linux-image-generic: /usr/share/doc/linux-image-generic

ceph

1
1. Check or watch cluster health: ceph status || ceph -w
If you want to quickly verify that your cluster is operating normally, use ceph status to get a birds-eye view of cluster status (hint: typically, you want your cluster to be active + clean). You can also watch cluster activity in real-time with ceph -w; you'll typically use this when you add or remove OSDs and want to see the placement groups adjust.

2. Check cluster usage stats: ceph df
To check a cluster’s data usage and data distribution among pools, use ceph df. This provides information on available and used storage space, plus a list of pools and how much storage each pool consumes. Use this often to check that your cluster is not running out of space.

3. Check placement group stats: ceph pg dump
When you need statistics for the placement groups in your cluster, use ceph pg dump. You can get the data in JSON as well in case you want to use it for automatic report generation.

4. View the CRUSH map: ceph osd tree
Need to troubleshoot a cluster by identifying the physical data center, room, row and rack of a failed OSD faster? Use ceph osd tree, which produces an ASCII art CRUSH tree map with a host, its OSDs, whether they are up and their weight.

5. Create or remove OSDs: ceph osd create || ceph osd rm
Use ceph osd create to add a new OSD to the cluster. If no UUID is given, it will be set automatically when the OSD starts up. When you need to remove an OSD from the CRUSH map, use ceph osd rm with the UUID.

6. Create or delete a storage pool: ceph osd pool create || ceph osd pool delete
Create a new storage pool with a name and number of placement groups with ceph osd pool create. Remove it (and wave bye-bye to all the data in it) with ceph osd pool delete.

7. Repair an OSD: ceph osd repair
Ceph is a self-repairing cluster. Tell Ceph to attempt repair of an OSD by calling ceph osd repair with the OSD identifier.

8. Benchmark an OSD: ceph tell osd.* bench
Added an awesome new storage device to your cluster? Use ceph tell to see how well it performs by running a simple throughput benchmark. By default, the test writes 1 GB in total in 4-MB increments.

9. Adjust an OSD’s crush weight: ceph osd crush reweight
Ideally, you want all your OSDs to be the same in terms of thoroughput and capacity...but this isn't always possible. When your OSDs differ in their key attributes, use ceph osd crush reweight to modify their weights in the CRUSH map so that the cluster is properly balanced and OSDs of different types receive an appropriately-adjusted number of I/O requests and data.

10. List cluster keys: ceph auth list
Ceph uses keyrings to store one or more Ceph authentication keys and capability specifications. The ceph auth list command provides an easy way to to keep track of keys and capabilities

-=DRAFT=-

List des auth pour copier la clef

1
root@cephmngnt-01:~# ceph auth list
root@cephmngnt-01:~# cat /etc/ceph/ceph.client.admin.keyring 
root@opennebula-01:~# cat /etc/ceph/ceph.client.admin.keyring

Check info disk

1
root@kvmserver-01:~# cat /etc/libvirt/qemu/client-bdd01.xml

List des disks

1
root@opennebula-01:~# rbd -p opennebula ls
root@opennebula-01:~# qemu-img info rbd:opennebula/one-666-1664-0

Rm et cp

1
root@opennebula-01:~# rbd rm opennebula/one-666-1664-0
root@opennebula-01:~# rbd cp libvirtpool/client-bdd01 opennebula/one-666-1664-0

Jimmy’s TIPS

1
- Visualisation de la configuration des OSD :
ceph -n osd.ID --show-config

- Changement de la taille du réplica et du min_size
ceph osd pool set POOOL size 2
ceph osd pool set POOOL min_size 2

- Augmentation du nombre de Placement Group :
ceph osd pool set NOM_DU_POOL pg_num 64
ceph osd pool set NOM_DU_POOL pgp_num 64

- Visualisation des performances de l'OSD
ceph daemon osd.ID perf dump

- Historique des dernières opérations (I/O) de l'OSD :
ceph daemon osd.ID dump_historic_ops

- Modification de la crush Map :
Cf slide

- Voir l'ensemble de PG
ceph pg dump

- Voir les informations lié à un PH
ceph pg ID query

- Sortir un osd :
ceph osd out ID

- Ajouter un osd déjà existant :
ceph osd in ID

- Mise en place de la maintenance et désactivation de la maintenance :
ceph osd set noout
ceph osd unset noout

- Mirroring RBD
http://docs.ceph.com/docs/jewel/rbd/rbd-mirroring/

- Process de MAJ
http://docs.ceph.com/docs/master/releases

Disk

scan to search for new device

1
# echo  1> /sys/class/block/sdb/device/rescan
# e2fsck -fy /dev/sdb
# resize2fs /dev/sdb
1
old
#  rescan-scsi-bus

Installé via la paquet : sg3_util ou scsitools

clamav

Whitelist

./eicar.com: Win.Test.EICAR_HSB-1 FOUND

1
echo "Win.Test.EICAR_HSB-1" >> /var/lib/clamav/whitelist.ign2

Un script de scan :

1
#!/usr/bin/env bash

# Scan and report for clamav
# by Jerem
# 20170313

TODAY=$(date +%Y.%m.%d)
LOGFILE=/var/log/clamav/clamav_scan.$TODAY.log
HOSTNAME=$(hostname)
#       --exclude=REGEX, --exclude-dir=REGEX

if [ -f $LOGFILE ]
        then rm -f $LOGFILE
fi

date > $LOGFILE
 
/usr/bin/clamscan -rio /data -l $LOGFILE

date >> $LOGFILE

if ((`/bin/grep Infected $LOGFILE | cut -d : -f2` > 0 ))
        then cat $LOGFILE | mail -s "Antivirus scan on $HOSTNAME" -a "From:supervision@plop.com" jerem@mail.fr
fi

Bench

Bench CPU via dd :

Pour 4 proc

1
fulload() { dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null | dd if=/dev/zero of=/dev/null & }; fulload; read; killall dd

Alerta

DOC : http://docs.alerta.io/en/latest/

1
apt-get install postfix
apt-get install mongodb
apt-get install python-pip
pip install --upgrade pip
pip install alerta-server
1
export ALERTA_SVR_CONF_FILE=/etc/alertad.conf
mkdir /var/log/alerta/
1
cat /etc/alertad.conf

# Conf Alerta
# http://docs.alerta.io/en/latest/configuration.html

# BASIC
DEBUG = False
BASE_URL = ''
LOGGER_NAME = 'alerta'
LOG_FILE = '/var/log/alerta/alertad.log'

# MONGODB
MONGO_URI = 'mongodb://localhost:27017/monitoring'
MONGO_DATABASE = 'monitoring'

# CORS
CORS_ORIGINS = [
    'http://127.0.0.1',
    'http://localhost',
    'http://192.168.50.45:8000',
    'http://192.168.50.45',
    'http://192.168.50.40'
]

# AUTH
AUTH_REQUIRED = False
SECRET_KEY = ''

ADMIN_USERS = ['astreinte@mail.fr']
CUSTOMER_VIEWS = False

#OAUTH2_CLIENT_ID = None  # Google or GitHub OAuth2 client ID and secret
#OAUTH2_CLIENT_SECRET = None
#ALLOWED_EMAIL_DOMAINS = ['*']
#ALLOWED_GITHUB_ORGS = ['*']
#
#GITLAB_URL = None
#ALLOWED_GITLAB_GROUPS = ['*']

TOKEN_EXPIRE_DAYS = 7

# API
QUERY_LIMIT = 10000
HISTORY_LIMIT = 100
API_KEY_EXPIRE_DAYS = 365

AUTO_REFRESH_ALLOW = 'ON'
SENDER_API_ALLOW = 'ON'

# EMAIL
EMAIL_VERIFICATION = False
SMTP_HOST = 'localhost'
SMTP_PORT = 25
MAIL_FROM = 'alerta@plop.com'
SMTP_PASSWORD = ''

# SUP
BLACKOUT_DURATION = 86400

SEVERITY_MAP = {
    'security': 0,
    'critical': 1,
    'major': 2,
    'minor': 3,
    'warning': 4,
    'indeterminate': 5,
    'cleared': 5,
    'normal': 5,
    'ok': 5,
    'informational': 6,
    'debug': 7,
    'trace': 8,
    'unknown': 9
}
DEFAULT_SEVERITY = 'indeterminate'

# Plug-ins
PLUGINS = ['reject','amqp','alerta-mailer']

# amqp https://github.com/alerta/alerta-contrib/tree/master/plugins/amqp
AMQP_URL = 'mongodb://localhost:27017/kombu'
AMQP_TOPIC = 'notify'

ORIGIN_BLACKLIST = ['foo/bar$', '.*/qux']  # reject all foo alerts from bar, and everything from qux
ALLOWED_ENVIRONMENTS = ['blabla']  # reject alerts without allowed environments
1
cat alertad.sh 
#!/bin/bash


### BEGIN INIT INFO
# Provides: alertad
# Required-Start: $remote_fs $all
# Should-Start:
# Required-Stop:
# Default-Start: 2 3 5
# Default-Stop: 0 6
# Description: Starts and stops alertad
### END INIT INFO


ABIN=/usr/local/bin/alertad


case $1 in

# Startup alertad
        start)

                $ABIN &
                echo "starting..."
        ;;

# Stop alertad
        stop)
                killall alertad
                echo "stopping..."
        ;;

        *)
                echo "stop ou start c'est tout"
        ;;
esac

alerta webui

DOC : http://docs.alerta.io/en/latest/webui.html

1
cd /opt
git clone https://github.com/alerta/angular-alerta-webui
cd /opt/angular-alerta-webui/app/
1
cat config.js

'use strict';

angular.module('config', [])
  .constant('config', {
    'endpoint'    : "http://"+window.location.hostname+":8080",
    'provider'    : "basic", // google, github, gitlab or basic
//    'client_id'   : "",
//    'gitlab_url'  : "https://gitlab.com",  // replace with your gitlab server
    'colors'      : {}, // use default colors
//    'colors'      : {
//      'severity': {
//        'critical'     : '#D8122A',
//        'major'        : '#EA680F',
//        'minor'        : '#FFBE1E',
//        'warning'      : '#BA2222',
//        'indeterminate': '#A6ACA8',
//        'cleared'      : '#00AA5A',
//        'normal'       : '#00AA5A',
//        'ok'           : '#00AA5A',
//        'informational': '#00A1BC',
//        'debug'        : '#9D006D',
//        'security'     : '#333333',
//        'unknown'      : '#A6ACA8'
//      },
//      'text': 'white',
//      'highlight': 'lightgray'
//    },
    'severity'    : {}, // use default severity codes
//    'severity'    : {
//      'fatal': 0
//    },
    'audio'       : {}, // no audio
//    'audio'       : {
//      'new'  : '/audio/Bike Horn.mp3'
//    }
//    'tracking_id' : ""  // Google Analytics tracking ID eg. UA-NNNNNN-N
  });
1
cat /etc/init.d/alerta-webui.sh
#!/bin/bash


### BEGIN INIT INFO
# Provides: alerta-webui
# Required-Start: $remote_fs $all
# Should-Start:
# Required-Stop:
# Default-Start: 2 3 5
# Default-Stop: 0 6
# Description: Starts and stops alerta-webui
### END INIT INFO


AWDIR=/opt/angular-alerta-webui/app/


case $1 in

# Startup alerta-webui
        start)

                cd $AWDIR && python -m SimpleHTTPServer 8000 &
                echo "starting..."
        ;;

# Stop alerta-webui
        stop)
                for i in  ` ps auxw | grep SimpleHTTPServer | grep -v grep | /usr/bin/awk '{print $2}'` ; do kill -9 $i ; done
                echo "stopping..."
        ;;

        *)
                echo "stop ou start c'est tout"
        ;;
esac

alerta mailer (& alerta-amqp)

Doc : https://github.com/alerta/alerta-contrib/tree/master/integrations/mailer

1
git clone https://github.com/alerta/alerta-contrib.git
cd /opt/alerta-contrib/plugins/amqp
python setup.py install
1
cd /opt/alerta-contrib/integrations/mailer
vi mailer.py
-    'config_file':   '~/.alerta.conf',
+    'config_file':   '/etc/alerta-mailer.conf',
1
python setup.py install
1
cat /etc/alerta-mailer.conf

[alerta-mailer]
key = 
mail_to = astreinte@mail.fr
mail_from = alerta@plop.com
dashboard_url = http://192.168.50.45:8000
endpoint =  http://192.168.50.45:8080
smtp_host = 127.0.0.1
smtp_port = 25
smtp_password = 
smtp_use_ssl = False
debug = True
skip_mta = False
email_type = html
mail_localhost = plop.com
amqp_url = mongodb://127.0.0.1:27017/kombu
amqp_topic = notify
mail_template = /opt/alerta-contrib/integrations/mailer/email.tmpl
mail_template_html = /opt/alerta-contrib/integrations/mailer/email.html.tmpl

alerta-mailer

1
cat /etc/cron.d/alerta-mailer_restart 
0 */2 * * * root /etc/init.d/alerta-mailer-init.sh stop && /bin/sleep 2 && /etc/init.d/alerta-mailer-init.sh start >> /var/log/alerta/restart 2>&1
1
cat /etc/init.d/alerta-mailer-init.sh

#!/bin/bash


### BEGIN INIT INFO
# Provides: alerta-mailer
# Required-Start: $remote_fs $all
# Should-Start:
# Required-Stop:
# Default-Start: 2 3 5
# Default-Stop: 0 6
# Description: Starts and stops alerta-mailer.
### END INIT INFO


AMBIN=/usr/local/bin/alerta-mailer


case $1 in

# Startup alerta-mailer
        start)

                $AMBIN &
                echo "starting..."
        ;;

# Stop alerta-mailer
        stop)
                killall alerta-mailer
                echo "stopping..."
        ;;

        *)
                echo "stop ou start c'est tout"
        ;;
esac

Docker

DOCKER

  • INTRODUCTION

registry publique

https://hub.docker.com/

Orchestration

compose(single)/swarm(multi host)
http://mesos.apache.org/

IMAGES manipulation

registry

1
~# docker search ubuntu
~# docker pull ubuntu:latest
~# docker push jeba/ubuntu

tag (rename)

1
~# docker tag jeba/ubuntu jeba/plop

details

1
~# docker inspect jeba/ubuntu | jq

run

daemon

1
~# docker run -d

interactiv (stdin) -t terminal

1
~# docker run -it --name myubuntu ubuntu bash

–rm : rm on stop
–restart always

exit whithout kill

ctrl+p+q

re attach

1
~# docker container attach [ID]

execute

1
~# docker container exec -i -t [id] <command>

logs

1
~# docker container logs
~# docker container logs -f
~# docker container logs --tail 20

stop/kill

1
~# docker container stop    (sigterm)
~# docker container kill    (sigkill)

bind

1
~# docker container run -d -p 80:80 nginx
~# docker container exec -it nginx bash

commit (save)

1
~# docker container commit nginx mon_nginx:0.1

cleaning

1
~# docker container prune
~# docker image prune

dockerfile

https://docs.docker.com/engine/reference/builder/

FROM : image de base
RUN : modif du fs
COPY : cp from host to container
ADD : cp also from an URL
ENTRYPOINT : run at start of the container
CMD : run at start of the container (can be overridden)( parameters for ENTRYPOINT (if exist))
ENV : set env variables
ARG : set variables
USER : set user
WORKDIR : set pwd
VOLUME
EXPOSE
SHELL
LABEL
HEALTHCHECK
STOPSIGNAL

Build image from dockerfile

1
~# docker image build -t name:tag <context_path> ( -f <file>  if not "Dockerfile" )
1
~# cat Dockerfile
FROM ubuntu:latest
RUN apt-get update  && \
    apt-get install iputils-ping -y
VOLUME /var/www
ENTRYPOINT ["ping"]
CMD ["8.8.8.8"]

VOLUMES

1
~# docker container run -v /home/wam:/container/home/
~# docker container run -v /home/wam:/container/home/:ro

~# docker volume create voltest  (--driver local --opt type=btrfs --opt device=/dev/sda2)  (OR --driver local --opt type=tmpfs --opt device=tmpfs --opt o=size=100m,uid=1000)
~# ~# ls /var/lib/docker/volumes/voltest/
~# docker container run -v voltest:/container/home/

NETWORK

map to host (host:container)

1
~# docker container run -d -p 80:80 nginx

auto map (on random port)

1
~# docker container run -d -P nginx

EXPOSE in dockerfile is informative !

1
~# docker network create --driver bridge mynetwork
~# docker network create --driver bridge --subnet 10.0.0.1/24 mynetwork

no dns resolv on docker0 (auto create bridge)
dns resolv auto activate on manually created bridge

ex:

1
~# docker network create -d bridge --subnet 192.168.69.0/24 testnet
~# docker run -d -P --network=testnet jeba/nginx:0.1

ENV VAR

export DOCKER_TLS_VERIFY=”1”
export DOCKER_HOST=”tcp://127.0.0.1:2375”
export DOCKER_CERT_PATH=”/home/jerem/.docker/certs”
export DOCKER_MACHINE_NAME=”myDock”

TIPS

Info

1
~# docker system

Clean

1
~# docker system prune

Exit/detach

1
~# ctrl+p+q

rm all

1
~# docker rm -f $(docker ps -aq)

Expose dockerengine port and use insecure registry

1
~# cat /lib/systemd/system/docker.service
 ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375
 ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 -insecure-registry myregistry:5000
 ExecStart=/usr/bin/dockerd -H fd:// -H tcp://0.0.0.0:2375 --tlsverify --tlscacert=/etc/docker/certs.d/ca.pem --tlscert=/etc/docker/certs.d/server-cert.pem --tlskey=/etc/docker/certs.d/server-key.pem

Create stack = overlay network and services (on swarm with compose)

1
~# docker stack deploy -c compose.yml mystack

Traefik

load balancer
https://docs.traefik.io/

Portainer

Webgui for swarm (or not)

http://portainer.io/

Kubernetes

https://kubeapps.com/

https://github.com/kubernetes/helm
http://helm.sh/

https://github.com/kubernetes/minikube

Rancher

http://rancher.com/

Others

https://www.nomadproject.io/
https://www.terraform.io/docs/providers/docker/index.html

List 10 last versions of an image on registry

curl ‘https://registry.hub.docker.com/v2/repositories/heketi/heketi/tags/'|jq ‘.”results”[][“name”]’

Raid

Copy table

1
sfdisk -d /dev/sdb | sfdisk -f /dev/sda

Create raid

1
mdadm --create --verbose /dev/md2 --level=1 --raid-devices=2 /dev/sdc /dev/sdd

Add disk to raid

1
mdadm --manage /dev/md0 --add /dev/sda1

Speed up rebuild

1
echo 50000 > /proc/sys/dev/raid/speed_limit_min

SSH

Escape sequences

1
~# ~?
Supported escape sequences:
 ~.   - terminate connection (and any multiplexed sessions)
 ~B   - send a BREAK to the remote system
 ~C   - open a command line
 ~R   - request rekey
 ~V/v - decrease/increase verbosity (LogLevel)
 ~^Z  - suspend ssh
 ~#   - list forwarded connections
 ~&   - background ssh (when waiting for connections to terminate)
 ~?   - this message
 ~~   - send the escape character by typing it twice
(Note that escapes are only recognized immediately after newline.)

restrict ssh acces

1
~# grep AllowUsers /etc/ssh/sshd_config
AllowUsers root@93.14.29.25 jerem none

use keepalive

1
ssh -o ServerAliveInterval=5 -o ServerAliveCountMax=1 10.0.0.1

ssh tunnel

  • local tunnel

-L : local
-f : background
-n : redirect stdin > /dev/null

1
~# ssh -L 8080:localhost:80 jerem.unvrai.info -f -n
~# curl http://localhost:8080
<h1> unvrai.info welcome page </h1>
1
~# ssh -L 8080:web.unvrai.info:80 jerem.unvrai.info -f -n
~# curl http://localhost:8080
<h1> web.unvrai.info home page </h1>
  • remote tunnel
1
~# ssh -R 8080:localhost:80 jerem@unvrai.info -f -n
~# curl http://unvrai.info:8080
<h1> my local welcome page </h1>
  • SSH connection through host in the middle

    1
    ~# ssh -t reachable_host ssh unreachable_host
  • Testing remote connection to port

    1
    timeout 1 bash -c "</dev/<proto>/<host>/<port>" >/dev/null 2>&1 ; echo $?

sshfs

1
~$ mkdir /opt/doc/ansible
~$ sshfs jerem@server.fr:doc/ansible /opt/doc/ansible
1
~# grep sshfs /etc/fstab
user@machine:/répertoire/distant                /mnt/mon_rep          fuse.sshfs           port=22,user,noatime,_netdev     0 0

Forward X

1
~# grep X11 /etc/ssh/sshd_config
X11Forwarding yes
#X11DisplayOffset 10
X11UseLocalhost no

Clean keys

1
~# ssh-keygen -f "/home/jerem/.ssh/known_hosts" -R 10.0.0.10