Ssl
List ciphers
1 | nmap --script ssl-enum-ciphers -p 443 localhost |
Gluster
Install
add-apt-repository ppa:gluster/glusterfs-3.10
apt-get update && apt-get install glusterfs-server -y
Doc
http://docs.gluster.org/en/latest/Quick-Start-Guide/Quickstart/
Conf
Create cluster1
root@jeb-gluster-01:~# gluster peer probe jeb-gluster-02
root@jeb-gluster-01:~# gluster peer probe jeb-gluster-03
Show status1
~# gluster peer status
Disks setup1
mkfs.xfs /dev/vdb
mkdir -p /data
mount /dev/vdb /data
mkdir /data/disk0
Create du vol1
( gluster volume create gv0 replica 3 arbiter 1 transport tcp jeb-gluster-01:/data/gv0 jeb-gluster-02:/data/gv0 jeb-gluster-03:/data/gv0 )
gluster volume create vol0 disperse 3 redundancy 1 transport tcp jeb-gluster-01:/data/disk0 jeb-gluster-02:/data/disk0 jeb-gluster-03:/data/disk0
gluster volume start vol0
Mount1
mkdir -p /mnt/vol0
mount -t glusterfs jeb-gluster-02:/vol0 /mnt/vol0
mount -t glusterfs -o backupvolfile-server=jeb-gluster-03,use-readdirp=no,log-level=WARNING,log-file=/var/log/gluster.log jeb-gluster-02:/vol0 /mnt/vol0
un peu de tuning
gluster volume set vol0 performance.cache-size 128MB
Troubleshooting
Luks
Luks
Install
1 | # apt-get update && apt-get install -y cryptsetup |
set up
Generate secret1
# openssl rand -base64 32 > /root/.luks
# chmod 400 /root/.luks
Create partition1
# cryptsetup luksFormat -c aes-xts-plain64 -s 512 -h sha512 /dev/vdb
Show informations1
# cryptsetup luksDump /dev/vdb
Uncrypt / crypt1
# cryptsetup luksOpen /dev/vdb safedata
# cryptsetup -v luksClose safedata
Format1
# mkfs.ext4 /dev/mapper/safedata
Associate key to crypted partition1
# cryptsetup -v luksAddKey /dev/vdb /root/.luks
Mount
1 | # cat /etc/crypttab safedata UUID=3ffa60f1-a93d-1e4f-9f82-4c7db85e6b3e /root/.luks luks |
1 | # cat /etc/fstab /dev/mapper/safedata /datas ext4 defaults 0 1 |
tmux
tmux shortcuts & cheatsheet
start new:
tmux
start new with session name:
tmux new -s myname
attach:
tmux a # (or at, or attach)
attach to named:
tmux a -t myname
maximize:
tmux a z
list sessions:
tmux ls
tmux kill-session -t myname
tmux ls | grep : | cut -d. -f1 | awk '{print substr($1, 0, length($1)-1)}' | xargs kill
In tmux, hit the prefix ctrl+b (my modified prefix is ctrl+a) and then:
Sessions
:new<CR> new session
s list sessions
$ name session
Windows (tabs)
c create window
w list windows
n next window
p previous window
f find window
, name window
& kill window
Panes (splits)
% vertical split
" horizontal split
o swap panes
q show pane numbers
x kill pane
+ break pane into window (e.g. to select text by mouse to copy)
- restore pane from window
⍽ space - toggle between layouts
<prefix> q (Show pane numbers, when the numbers show up type the key to goto that pane)
<prefix> { (Move the current pane left)
<prefix> } (Move the current pane right)
<prefix> z toggle pane zoom
Sync Panes
You can do this by switching to the appropriate window, typing your Tmux prefix (commonly Ctrl-B or Ctrl-A) and then a colon to bring up a Tmux command line, and typing:
1 | :setw synchronize-panes |
You can optionally add on or off to specify which state you want; otherwise the option is simply toggled. This option is specific to one window, so it won’t change the way your other sessions or windows operate. When you’re done, toggle it off again by repeating the command. tip source
Resizing Panes
You can also resize panes if you don’t like the layout defaults. I personally rarely need to do this, though it’s handy to know how. Here is the basic syntax to resize panes:
PREFIX : resize-pane -D (Resizes the current pane down)
PREFIX : resize-pane -U (Resizes the current pane upward)
PREFIX : resize-pane -L (Resizes the current pane left)
PREFIX : resize-pane -R (Resizes the current pane right)
PREFIX : resize-pane -D 20 (Resizes the current pane down by 20 cells)
PREFIX : resize-pane -U 20 (Resizes the current pane upward by 20 cells)
PREFIX : resize-pane -L 20 (Resizes the current pane left by 20 cells)
PREFIX : resize-pane -R 20 (Resizes the current pane right by 20 cells)
PREFIX : resize-pane -t 2 20 (Resizes the pane with the id of 2 down by 20 cells)
PREFIX : resize-pane -t -L 20 (Resizes the pane with the id of 2 left by 20 cells)
Copy mode:
Pressing PREFIX [ places us in Copy mode. We can then use our movement keys to move our cursor around the screen. By default, the arrow keys work. we set our configuration file to use Vim keys for moving between windows and resizing panes so we wouldn’t have to take our hands off the home row. tmux has a vi mode for working with the buffer as well. To enable it, add this line to .tmux.conf:
setw -g mode-keys vi
With this option set, we can use h, j, k, and l to move around our buffer.
To get out of Copy mode, we just press the ENTER key. Moving around one character at a time isn’t very efficient. Since we enabled vi mode, we can also use some other visible shortcuts to move around the buffer.
For example, we can use “w” to jump to the next word and “b” to jump back one word. And we can use “f”, followed by any character, to jump to that character on the same line, and “F” to jump backwards on the line.
Function vi emacs
Back to indentation ^ M-m
Clear selection Escape C-g
Copy selection Enter M-w
Cursor down j Down
Cursor left h Left
Cursor right l Right
Cursor to bottom line L
Cursor to middle line M M-r
Cursor to top line H M-R
Cursor up k Up
Delete entire line d C-u
Delete to end of line D C-k
End of line $ C-e
Goto line : g
Half page down C-d M-Down
Half page up C-u M-Up
Next page C-f Page down
Next word w M-f
Paste buffer p C-y
Previous page C-b Page up
Previous word b M-b
Quit mode q Escape
Scroll down C-Down or J C-Down
Scroll up C-Up or K C-Up
Search again n n
Search backward ? C-r
Search forward / C-s
Start of line 0 C-a
Start selection Space C-Space
Transpose chars C-t
Misc
d detach
t big clock
? list shortcuts
: prompt
+ maximize/minimize pane
Configurations Options:
# Mouse support - set to on if you want to use the mouse
* setw -g mode-mouse off
* set -g mouse-select-pane off
* set -g mouse-resize-pane off
* set -g mouse-select-window off
# Set the default terminal mode to 256color mode
set -g default-terminal "screen-256color"
# enable activity alerts
setw -g monitor-activity on
set -g visual-activity on
# Center the window list
set -g status-justify centre
# Maximize and restore a pane
unbind Up bind Up new-window -d -n tmp \; swap-pane -s tmp.1 \; select-window -t tmp
unbind Down
bind Down last-window \; swap-pane -s tmp.1 \; kill-window -t tmp
Resources:
Galera / Haproxy
1 | mybdd:~# CREATE USER 'haproxy'@'192.168.1.10'; |
1 | # tail /etc/haproxy/haproxy.cfg listen galera 192.168.1.10:3306 balance source mode tcp option tcpka option mysql-check user haproxy server node1 192.168.1.1:3306 check weight 1 server node2 192.168.1.2:3306 check weight 1 server node2 192.168.1.3:3306 check weight 1 |
vault
-=DRAFT=-
1 | vault$ cat config.hcl #storage "consul" { # address = "127.0.0.1:8500" # scheme = "https" # tls_ca_file = "/etc/pem/vault.ca" # tls_cert_file = "/etc/pem/vault.cert" # tls_key_file = "/etc/pem/vault.key" # path = "vault" # token = "abcd1234" #} storage "file" { path = "/var/lib/vault/data" } listener "tcp" { address = "127.0.0.1:8200" tls_disable = 1 # tls_cert_file = "/etc/vault/nomad.crt" # tls_key_file = "/etc/vault/nomad.key" } telemetry { statsite_address = "127.0.0.1:8125" disable_hostname = true } |
1 | # export VAULT_ADDR='http://127.0.0.1:8200' |
Kubernetes
DRAFT
========== KUKU BERNADETTE =========
1 | # Gestion certif wget https://pkg.cfssl.org/R1.2/cfssl_linux-amd64 chmod +x cfssl_linux-amd64 mv cfssl_linux-amd64 /usr/local/bin/cfssl wget https://pkg.cfssl.org/R1.2/cfssljson_linux-amd64 chmod +x cfssljson_linux-amd64 mv cfssljson_linux-amd64 /usr/local/bin/cfssljson mkdir /root/certs ; cd /root/certs echo '{ "signing": { "default": { "expiry": "8760h" }, "profiles": { "kubernetes": { "usages": ["signing", "key encipherment", "server auth", "client auth"], "expiry": "8760h" } } } }' > ca-config.json echo ' { "CN": "jay-cluster", "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "FR", "L": "Paris", "O": "JayCorp", "OU": "IT", "ST": "IDF" } ] }' > ca-csr.json cfssl gencert -initca ca-csr.json | cfssljson -bare ca echo '{ "CN": "jay-cluster", "hosts": [ "10.0.45.15", "10.0.45.28", "10.0.45.31", "jeb-test-3740.one.ippon-hosting.net", "jeb-test1-3741.one.ippon-hosting.net", "jeb-test2-3742.one.ippon-hosting.net", "jeb-test-3740", "jeb-test1-3741", "jeb-test2-3742", "localhost", "10.32.0.1", "127.0.1.1", "127.0.0.1" ], "key": { "algo": "rsa", "size": 2048 }, "names": [ { "C": "FR", "L": "Paris", "O": "JayCorp", "OU": "IT", "ST": "IDF" } ] }' > kubernetes-csr.json cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes kubernetes-csr.json | cfssljson -bare kubernetes scp *pem 10.0.45.28:/root/certs/ scp *pem 10.0.45.31:/root/certs/ # Etcd https://github.com/coreos/etcd/releases/latest/ groupadd etcd useradd etcd -d /var/lib/etcd -s /bin/false -g etcd mkdir -p /var/lib/etcd mkdir -p /etc/etcd/ cp /root/certs/ca.pem /root/certs/kubernetes-key.pem /root/certs/kubernetes.pem /etc/etcd/ chown etcd:etcd /var/lib/etcd -R chown etcd:etcd /etc/etcd -R wget https://github.com/coreos/etcd/releases/download/v3.2.9/etcd-v3.2.9-linux-amd64.tar.gz tar -xvf etcd-v3.2.9-linux-amd64.tar.gz mv etcd-v3.2.9-linux-amd64/etcd* /usr/local/bin/ echo '[Unit] Description=etcd Documentation=https://github.com/coreos [Service] User=etcd ExecStart=/usr/local/bin/etcd --name jeb-test-3740 \ --cert-file=/etc/etcd/kubernetes.pem \ --key-file=/etc/etcd/kubernetes-key.pem \ --peer-cert-file=/etc/etcd/kubernetes.pem \ --peer-key-file=/etc/etcd/kubernetes-key.pem \ --trusted-ca-file=/etc/etcd/ca.pem \ --peer-trusted-ca-file=/etc/etcd/ca.pem \ --initial-advertise-peer-urls https://10.0.45.15:2380 \ --listen-peer-urls https://10.0.45.15:2380 \ --listen-client-urls https://10.0.45.15:2379,http://127.0.0.1:2379 \ --advertise-client-urls https://10.0.45.15:2379 \ --initial-cluster-token etcd-jay-cluster \ --initial-cluster jeb-test-3740=https://10.0.45.15:2380,jeb-test1-3741=https://10.0.45.28:2380,jeb-test2-3742=https://10.0.45.31:2380 \ --initial-cluster-state new \ --data-dir=/var/lib/etcd Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target' > etcd.service mv etcd.service /lib/systemd/system/ systemctl daemon-reload systemctl enable etcd systemctl start etcd etcdctl --ca-file=/etc/etcd/ca.pem cluster-health # Flanneld https://github.com/coreos/flannel/releases/ etcdctl --ca-file=/etc/etcd/ca.pem set /flanneld/network/config '{ "Network": "10.32.0.0/16" }' wget https://github.com/coreos/flannel/releases/download/v0.9.0/flannel-v0.9.0-linux-amd64.tar.gz tar -zxpvf flannel-v0.9.0-linux-amd64.tar.gz mv flanneld /usr/local/bin/ mkdir -p /var/lib/kubernetes/flannel/networks echo '[Unit] Description=Network fabric for containers Documentation=https://github.com/coreos/flannel After=etcd.service [Service] Type=notify Restart=always RestartSec=5 ExecStart=/usr/local/bin/flanneld \ -etcd-endpoints=https://10.0.45.15:2379 \ -etcd-keyfile=/etc/etcd/kubernetes-key.pem \ -etcd-certfile=/etc/etcd/kubernetes.pem \ -etcd-cafile=/etc/etcd/ca.pem \ -etcd-prefix=/flanneld/network \ -subnet-file=/var/lib/kubernetes/flannel/subnet.env [Install] WantedBy=multi-user.target' > /lib/systemd/system/flanneld.service systemctl daemon-reload systemctl enable flanneld systemctl start flanneld # Install docker apt-get update && apt-get install -y curl apt-transport-https curl -fsSL https://download.docker.com/linux/ubuntu/gpg | apt-key add - cat <<EOF >/etc/apt/sources.list.d/docker.list deb https://download.docker.com/linux/$(lsb_release -si | tr '[:upper:]' '[:lower:]') $(lsb_release -cs) stable EOF apt-get update && apt-get install -y docker-ce echo '[Unit] Description=Docker Application Container Engine Documentation=https://docs.docker.com After=network-online.target docker.socket firewalld.service flanneld.service Wants=network-online.target Requires=docker.socket [Service] Type=notify EnvironmentFile=-/var/lib/kubernetes/flannel/subnet.env ExecStart=/usr/bin/dockerd -H fd:// --bip=${FLANNEL_SUBNET} --mtu=${FLANNEL_MTU} ExecReload=/bin/kill -s HUP $MAINPID LimitNOFILE=1048576 LimitNPROC=infinity LimitCORE=infinity TasksMax=infinity TimeoutStartSec=0 Delegate=yes KillMode=process Restart=on-failure StartLimitBurst=3 StartLimitInterval=60s [Install] WantedBy=multi-user.target' > /lib/systemd/system/docker.service systemctl daemon-reload systemctl enable docker systemctl start docker # Kubernetes groupadd kube useradd kube -g kube -d /var/lib/kubernetes/ -s /bin/false gpasswd -a kube docker https://github.com/kubernetes/kubernetes/releases/ mkdir -p /var/lib/kubernetes cp /root/certs/ca.pem /root/certs/kubernetes-key.pem /root/certs/kubernetes.pem /var/lib/kubernetes/ chown -R kube:kube /var/lib/kubernetes wget https://storage.googleapis.com/kubernetes-release/release/v1.7.8/bin/linux/amd64/kube-apiserver wget https://storage.googleapis.com/kubernetes-release/release/v1.7.8/bin/linux/amd64/kube-controller-manager wget https://storage.googleapis.com/kubernetes-release/release/v1.7.8/bin/linux/amd64/kube-scheduler wget https://storage.googleapis.com/kubernetes-release/release/v1.7.8/bin/linux/amd64/kubectl wget https://storage.googleapis.com/kubernetes-release/release/v1.7.8/bin/linux/amd64/kubelet wget https://storage.googleapis.com/kubernetes-release/release/v1.7.8/bin/linux/amd64/kube-proxy chmod +x kube-apiserver kube-controller-manager kube-scheduler kubectl kube-proxy kubelet mv kube-apiserver kube-controller-manager kube-scheduler kubectl kube-proxy kubelet /usr/local/bin/ echo '[Unit] Description=Kubernetes API Server Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] User=kube ExecStart=/usr/local/bin/kube-apiserver \ --admission-control=NamespaceLifecycle,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota \ --advertise-address=10.0.45.15 \ --service-node-port-range=30000-32767 \ --allow-privileged=true \ --enable-swagger-ui=true \ --apiserver-count=3 \ --authorization-mode=ABAC,RBAC \ --authorization-policy-file=/var/lib/kubernetes/authorization-policy.jsonl \ --bind-address=0.0.0.0 \ --insecure-bind-address=0.0.0.0 \ --service-cluster-ip-range=10.32.0.0/16 \ --kubelet-certificate-authority=/var/lib/kubernetes/ca.pem \ --kubelet-client-certificate=/var/lib/kubernetes/kubernetes.pem \ --kubelet-client-key=/var/lib/kubernetes/kubernetes-key.pem \ --service-account-key-file=/var/lib/kubernetes/kubernetes-key.pem \ --tls-cert-file=/var/lib/kubernetes/kubernetes.pem \ --tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \ --tls-ca-file=/var/lib/kubernetes/ca.pem \ --token-auth-file=/var/lib/kubernetes/token.csv \ --etcd-servers=https://10.0.45.15:2379,https://10.0.45.28:2379,https://10.0.45.31:2379 \ --etcd-certfile=/var/lib/kubernetes/kubernetes.pem \ --etcd-keyfile=/var/lib/kubernetes/kubernetes-key.pem \ --etcd-cafile=/var/lib/kubernetes/ca.pem \ --storage-backend=etcd2 \ --storage-media-type=application/json \ --v=2 Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target' > kube-apiserver.service mv kube-apiserver.service /lib/systemd/system/ systemctl daemon-reload systemctl enable kube-apiserver echo '[Unit] Description=Kubernetes Controller Manager Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] User=kube ExecStart=/usr/local/bin/kube-controller-manager \ --allocate-node-cidrs=true \ --cluster-cidr=10.0.45.0/24 \ --cluster-name=jay-cluster \ --leader-elect=true \ --master=http://10.0.45.15:8080 \ --root-ca-file=/var/lib/kubernetes/ca.pem \ --service-account-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \ --service-cluster-ip-range=10.32.0.0/16 \ --v=2 Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target' > kube-controller-manager.service mv kube-controller-manager.service /lib/systemd/system/ systemctl daemon-reload systemctl enable kube-controller-manager echo '[Unit] Description=Kubernetes Scheduler Documentation=https://github.com/GoogleCloudPlatform/kubernetes [Service] User=kube ExecStart=/usr/local/bin/kube-scheduler \ --leader-elect=true \ --master=http://10.0.45.15:8080 \ --v=2 Restart=on-failure RestartSec=5 LimitNOFILE=65536 [Install] WantedBy=multi-user.target' > kube-scheduler.service mv kube-scheduler.service /lib/systemd/system/ systemctl daemon-reload systemctl enable kube-scheduler date +%s | sha256sum | base64 | head -c 32 ; echo echo 'MGY1ZTE1NTQyZjJjOTYzNjU4ZTJiZThj,admin,admin YzRlMWUwOGE2YjM0ZWRlN2UyNWNiODc3,kubelet,kubelet' > /var/lib/kubernetes/token.csv echo '{"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"*", "nonResourcePath": "*", "readonly": true}} {"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"admin", "namespace": "*", "resource": "*", "apiGroup": "*"}} {"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"user":"scheduler", "namespace": "*", "resource": "*", "apiGroup": "*"}} {"apiVersion": "abac.authorization.kubernetes.io/v1beta1", "kind": "Policy", "spec": {"group":"system:serviceaccounts", "namespace": "*", "resource": "*", "apiGroup": "*", "nonResourcePath": "*"}}' > /var/lib/kubernetes/authorization-policy.jsonl chown -R kube:kube /var/lib/kubernetes/* systemctl start kube-apiserver systemctl start kube-controller-manager systemctl start kube-scheduler kubectl get componentstatuses # Kube worker mkdir -p /var/lib/kubernetes echo '[Unit] Description=Kubernetes Kubelet Documentation=https://github.com/GoogleCloudPlatform/kubernetes After=docker.service flanneld.service Requires=docker.service [Service] ExecStart=/usr/local/bin/kubelet \ --allow-privileged=true \ --api-servers=https://10.0.45.15:6443,https://10.0.45.28:6443,https://10.0.45.31:6443 \ --cloud-provider= \ --cluster-dns=10.32.0.10 \ --cluster-domain=jaycorp \ --container-runtime=docker \ --docker=unix:///var/run/docker.sock \ --require-kubeconfig \ --kubeconfig=/var/lib/kubernetes/kubeconfig \ --client-ca-file=/var/lib/kubernetes/ca.pem \ --serialize-image-pulls=false \ --tls-cert-file=/var/lib/kubernetes/kubernetes.pem \ --tls-private-key-file=/var/lib/kubernetes/kubernetes-key.pem \ --cpu-cfs-quota=false \ --logtostderr=true \ --port=10250 \ --address=0.0.0.0 \ --pod-cidr=10.32.0.0/16 \ --chaos-chance=0.0 \ --v=2 Restart=on-failure RestartSec=5 [Install] WantedBy=multi-user.target' > /lib/systemd/system/kubelet.service systemctl daemon-reload systemctl enable kubelet echo '[Unit] Description=Kubernetes Proxy Documentation=https://github.com/kubernetes/kubernetes After=flanneld.service [Service] User=root ExecStart=/usr/local/bin/kube-proxy \ --master=http://10.0.45.15:8080 \ --logtostderr=true Restart=on-failure LimitNOFILE=65536 [Install] WantedBy=multi-user.target' > /lib/systemd/system/kube-proxy.service systemctl daemon-reload systemctl enable kube-proxy kubectl config set-credentials admin --token='MGY1ZTE1NTQyZjJjOTYzNjU4ZTJiZThj' --cluster='jay-cluster' --user='admin' --server='https://10.0.45.15:6443' kubectl config set-credentials admin --cluster='jay-cluste'r --client-certificate='/var/lib/kubernetes/kubernetes.pem' --client-key='/var/lib/kubernetes/kubernetes-key.pem' --certificate-authority='/var/lib/kubernetes/ca.pem' --server='https://10.0.45.15:6443' kubectl config set-cluster jay-cluster --certificate-authority='/var/lib/kubernetes/ca.pem' --embed-certs=true --server='https://10.0.45.15:6443' kubectl config set-context admin --cluster=jay-cluster --user=admin --server=https://10.0.45.15:6443 kubectl config use-context admin echo "apiVersion: v1 clusters: - cluster: `grep certificate-authority-data /root/.kube/config` server: https://10.0.45.15:6443 name: jay-cluster contexts: - context: cluster: jay-cluster user: admin name: admin current-context: admin kind: Config preferences: {} users: - name: admin user: token: MGY1ZTE1NTQyZjJjOTYzNjU4ZTJiZThj" > /var/lib/kubernetes/kubeconfig chown -R kube: /var/lib/kubernetes systemctl start kubelet systemctl start kube-proxy kubectl get nodes |
=======TIPS===========
kubectl taint nodes –all node-role.kubernetes.io/master-
etcdctl –ca-file=/etc/etcd/ca.pem cluster-health
kubectl proxy –address=’0.0.0.0’ –port=8001 –accept-hosts=’^*$’
kubectl config view
kubectl get componentstatuses
kubectl get pods –all-namespaces –show-all
kubectl get deployment –all-namespaces –show-all
kubectl get service –all-namespaces
kubectl get nodes
kubectl cluster-info
kubectl get events –namespace=kube-system
kubectl drain
kubectl delete node
source <(kubectl completion bash)
kubectl logs pod/skydns-352548417-bg8r5 skydns –namespace=kube-system
curl http://127.0.0.1:4194/validate/
curl 10.0.45.15:8080/healthz
kubectl cp /tmp/foo
=============================================