Unable to login via SSH with password - can login with ssh key and can su

This is because sshd has registered too many logins with the incorrect password.
To reset the incorrect password count, run the following command:

pam_tally2 --<user> --reset

This is configured under /etc/pam.d/sshd

Original article at:
http://www.tecmint.com/use-pam_tally2-to-lock-and-unlock-ssh-failed-login-attempts

Accurately determine the number of CPU cores for a Linux machine

The number of processors shown by /proc/cpuinfo might not be the actual number of cores on the processor. For example a processor with 2 cores and hyperthreading would be reported as a processor with 4 cores. To get the actual number of cores, check the core id for unique values as follows:

cat /proc/cpuinfo | grep "core id" | sort -u 

login to existing docker container as root

Run the following command from the host machine:

docker exec -itu 0 <container name or id> bash

Explanation:
The -i option means this session should be interactive
The -t option means attach the session to a tty
The -u 0 option means run the session as uid zero, which is of course root
And finally the command to run in this case is bash

Transferring Docker Images between host machines

Save the image on the source host:

docker save -o myimage.tar <image id or name>  

Now copy the saved image to the destination machine:

scp myimage.tar bob@jupiter.stack1.com:/docker-archives/myimage.tar

Then load the image:

docker load -i  /docker-archives/myimage.tar
or:
docker load < /docker-archives/myimage.tar

Nginx - alias vs root

Consider the following nginx config snippet:

server {
    listen       80;
    server_name  my_server;

    location /foo {
        root         /var/html;
        autoindex on;
        allow all;
    }
    location /bar {
        alias         /var/html;
        autoindex on;
        allow all;
    }
}

The root directive will map a request for http://myserver.stack1.com/foo to a directory /var/html/foo

The alias directive will map a request for http://myserver.stack1.com/bar to a directory /var/html  

(i.e. The rested URI is not appended to the file system location)

Gitflow workflow

My understanding of how this differs from simple feature branches:

1. To start with, two branches exist named master and develop (the dev branch is derived from master)
2. Developers create and work on feature branches from the develop branch in the usual way and these are merged back into the develop branch regularly via pull requests.
3. If using semantic versioning, the minor number (2nd field) is incremented each time a backwards compatible feature is added. A non backwards compatible feature (breaking feature) would result in the major number (1st field) being incremented. 
4. When it is decided that a release is due, a release branch is created from the develop branch. New features can still be added to the develop branch via feature branches in the usual way.
5. A deployable artifact is built from the release branch and the artifact is deployed to the appropriate test environments.
6. The deployed artifact is tested and any bug fixes are carried out on the release branch. Bug fixes increment the patch number (3rd field) of the semantic version number. NOTE: Only bug fixes can be carried out in the release branch. No features can be added to the release branch.
7. When all bugs are resolved and the release is finalised, the release branch is merged into both develop and master branches and master is also tagged with the semantic version number.

Official doc:  https://www.atlassian.com/git/tutorials/comparing-workflows

Installing a specific version of a yum package

using nodejs as an example

Search for the available versions:

sudo yum list nodejs --showduplicates

This returns something like:

nodejs.i686       0.10.48-3.el6
nodejs.x86_64     0.10.48-3.el6
nodejs.x86_64     5.11.1-1nodesource.el6

Therefore, to install version 5.11.1:

yum install nodejs-5.11.1-1nodesource.el6

As can be seen, it's a case of removing the extension from the package name (i.e nodejs.x86_64 becomes nodejs) and then concatenating the version by way of a hyphen, the result being  nodejs-5.11.1-1nodesource.el6

Finding out which yum package contains a specific binary

If I needed to install Apache Bench, the binary for which is ab
I would run:

yum provides ab

This returns:

Loading mirror speeds from cached hostfile
 * base: mirrors.clouvider.net
 * epel: epel.mirrors.ovh.net
 * extras: centos.serverspace.co.uk
 * updates: mirror.sov.uk.goscomb.net
httpd-tools-2.4.6-45.el7.centos.x86_64 : Tools for use with the Apache HTTP Server
Repo        : base
Matched from:
Filename    : /usr/bin/ab

Therefore the package needed is httpd-tools (sudo yum install http-tools)

Accessing yum from behind a corporate proxy

Append proxy and credentials to your /etc/yum.conf file as follows

proxy=http://proxy-host:proxy-port
proxy_username=username
proxy_password=password

Delete jenkins builds using curl

To delete a single job: 

curl -X POST http://jenkins-user:jenkins-token@hostname:port/job/execute-pipeline/26/doDelete

To delete a range of jobs: 

curl -X POST http://jenkins-user:jenkins-token@hostname:port/job/execute-pipeline/[20-30]/doDelete

This is handy for deleting a build arising from a git event you’ve chosen to ignore in your Jenkinsfile:

def buildId = env.BUILD_ID
node {
    sh "curl -X POST http://my_user:my_token@my_host:8080/job/execute-pipeline/${buildId}/doDelete"
}

Adding new RPM's to an existing yum repo

1.) Copy the rpm to the repository directory (i.e alongside all the existing rpm's)

2.) Update the repository index:

createrepo --update <path to repo> --verbose

3.) Make sure your file permissions for <path to repo>/repodata are ok (i.e. so that your http server can access the files):

chmod -R go+rx <path to repo>/repodata

(NOTE: don't forget to ensure the actual package file (the rpm) has the correct file permissions also)

4.) Clear your yum client cache:

yum clean all 

Set up a local yum repository

Install yum-utils

yum install yum-utils createrepo

Create a directory for the repo

mkdir /my_repo

Get the RPM's to put in the repo. Use reposync to download from an existing repo or download rpm's manaully.. e.g.

wget http://mirror.lihnidos.org/CentOS/6/os/i386/Packages/NetworkManager-0.8.1-43.el6.i686.rpm

Create the repository

createrepo /my_repo

Create /etc/yum.repos.d/custom.repo with the following contents

[customrepo]

name=Custom Repository
baseurl=file:////my_repo/
enabled=1
gpgcheck=0

PM2 notes

Passing Environment Variables

Create a file called process.json with the following contents

{
  "apps" : [{
    "name"        : "my-app",
    "script"      : "v1/app.js",
    "env": {
        "VERSION": 1.0.0
     }
  }]

}

Now run the app as follows

pm2 start process.json

The variable process.env.VERSION will now be available inside your nodejs application and will be set to the value 1.0.0

Authenticate to an internal npm repository

Run the following commands:

npm config set registry https://my_nexus_server/repositories/npm/
npm config set strict-ssl false
npm config set always-auth true
npm config set _auth bXktdXNlcjpteS1wYXNzd29yZA==

Generate the base64 as follows (don't forget the -n otherwise the newline character will be added to the base64 string:

echo -n 'my-user:my-password' | openssl base64

Alternatively you can just create a ~/.npmrc file as follows:

registry=https://my_nexus_server/repositories/npm/
strict-ssl=false
always-auth=true
_auth="bXktdXNlcjpteS1wYXNzd29yZA=="

NOTE: any config keys starting with an underscore are hidden, therefore they will not be echoed to the screen when running  npm config --list

Pull Docker images from behind a corporate proxy - TLS Trust

IMPORTANT NOTE: This applies to an old version of Docker (1.7) when using on RHEL 6.6 and therefore may not be relevant to newer versions. 

If you need to pull images from Docker Hub from behind a corporate proxy you will need to address two issues:

1. You will need to configure Docker to use (and potentially authenticate with) your corporate HTTP proxy
2. If you're receiving a message along the lines of "X509 certificate signed by an unrecognised CA" you will need to configure Docker to trust the corporate proxies TLS certificate. 

The first issue is addressed as follows:

• Open the file /etc/sysconfig/docker and add the following two lines

export HTTP_PROXY=http://username:password@proxy-hostname:proxy-port
export HTTPS_PROXY=http://username:password@proxy-hostname:proxy-port

The second issue is addressed as follows: 
(NOTE: in my experience using the --insecure-registry option does not work. This is contrary to the documentation and contrary to the advice given in the logs)

• Create a directory named /etc/docker/certs.d (I believe /etc/docker is the default and is configurable via the DOCKER_CERTS property which can be set in /etc/sysconfig/docker. However I haven't tested this)
• Under /etc/docker/certs.d, create a directory for each domain of the Docker registry you need to trust. For example if you're attempting to pull from index.docker.io, you should create a directory named index.docker.io. To be clear, the resulting path will be /etc/docker/certs.d/index.docker.io. (NOTE:  It's likely that multiple domains will need to be added. In order to determine this, see the section titled Debugging below)
• Copy a PEM formatted copy of your corporate proxy's certificate (or associated CA signer) into the directory(s) you created in the previous step. The filename used should be ca.crt
• Restart docker engine.
• Now re-attempt your docker pull and all should be well. If not, set your log level to debug as specified below and tail the logs to determine which directories need to be created (the logs are verbose enough in debug mode to point you in the right direction).

Debugging

To determine which domains are acting as Docker registries and subsequently which directories need to be created under /etc/docker/certs.d, set the log level for docker engine to debug as follows:

• Open /etc/sysconfig/docker for editing
• Add the --log-level=debug to other_args
• restart docker engine

Adding Linux users via ssh

bash script to add users via and set password via ssh

#!/bin/bash
 
USER_TO_ADD=ansible
USER_TO_ADD_UID=30000
USER_TO_ADD_GID=30000
 
[ $# -eq 1 ] || {
        echo "please provide target host"
        exit 1
}
 
TARGET_HOST=$1
R=$(ping ${TARGET_HOST} 2>&1)
[ $? -eq 0 ] || {
        echo "Unable to ping  ${TARGET_HOST}. Error is $R"
        exit 1
}
  
ssh -t ${TARGET_HOST} "sudo groupadd ${USER_TO_ADD} --gid ${USER_TO_ADD_GID}" && \
 "sudo useradd -m ${USER_TO_ADD} --uid ${USER_TO_ADD_GID} --gid ${USER_TO_ADD_GID}" && \
 "getent passwd ${USER_TO_ADD}" && \
 "usermod --groups wheel ${USER_TO_ADD}" && \
 "echo -e '${USER_TO_ADD}\n${USER_TO_ADD}\n' | sudo passwd ${USER_TO_ADD}"

storing git credentials

If you need to store git credentials so that you don't have to keep entering them when cloning a git repo over http(s)

Add the following to ~/.gitconfig

[user]
        email = bobmclarke@gmail.com
        name = Bob Clarke
[credential]
        helper = store

Then create ~/.git-credentials with the following contents

https://your-username:your-password@github.com

Now authentication to Git will work without prompting for credentials.

Another way of generating these files is to run:

git config --global user.email "bobmclarke@gmail.com"
git config --global user.name "Bob Clarke"

git config --global credential.helper store

Then push to the repo at which point you'll be prompted for your username and password. After you enter them for the first time a ~/.git-credentials file will be generated as above.

This can be useful if the developers in your team are using git as an npm repo for internal modules with something in the package.json like:

"some-npm-package-name": "git+https://github.com/bobclarke/mymodule#v1.0"

The downside is that your Git credentials will be stored in PLAIN TEXT so be aware.

Last thing to note, if you're doing this on an internal Github you may need to ignore ssl certs as follows:

git config --global http.sslVerify false