CategoryAdministration

Switch configuration lines using comments

Recently I have the case where I have to use a base Docker image from a remote or local repository. I love to keep the configuration close and to not have a couple of configuration files with the same content so I decided to write a simple program which will do the config switch and then will return it back (if needed)

Here is an example of usage:

When I build the images locally I want to use:

FROM anaconda-environment:latest

When I want to build from our CI/CD server I would like to use the remote ECR

CONFIG->remote-images:FROM XXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/anaconda-environment:latest

Then in the Docker file, I put those lines

# CONFIG->local-images:FROM anaconda-environment:latest
# CONFIG->remote-images:FROMXXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/anaconda-environment:latest
FROM anaconda-environment:latest
ENV ACCEPT_INTEL_PYTHON_EULA=yes
.... and so on...

And here is how the config is changed to point to the remote-images

ruby switch-config.rb Dockerfile remote-images

and this is how it is changed back to local images.

ruby switch.rb Dockerfile local-images

So far I haven’t found any drawback on this approach. For sure there is some drawbacks please let me know if you hit it.

And here is the code…

#!/usr/bin/env ruby

# ruby switch.rb Dockerfile remote-images
#
# # CONFIG->local-images:FROM anaconda-environment:latest
# # CONFIG->remote-images:FROM XXXXXXXXXXX.dkr.ecr.us-east-1.amazonaws.com/anaconda-environment:latest
# FROM anaconda-environment:latest




file = ARGV[0] || "Dockerfile"
desired_environment = ARGV[1] || "local"
lines = File.read(file).split("\n")
tag = /\s*?#\s+CONFIG->([\w\-]+?):(.*)\Z/
new_lines = []
key_found = true
options = {}
found_config_lines = false

lines.each do |line|
  if line =~ tag
    environment_key = $1
    option = $2
    options[environment_key] = option
    found_config_lines = true
  end

  if found_config_lines && options.values.include?(line)
    key_found = options.key?(desired_environment)
    new_lines << options[desired_environment]
    found_config_lines = false
    options = {}
  else
    new_lines << line
  end
end

if key_found
  File.write(file, new_lines.join("\n"))
else
  puts "Something is wrong - key not found! The syntax for defining options is:"
  puts "# CONFIG->KEY:The content of this option"
end

The meaning of “phlpwcspweb3” or why you should not do abbreviations in the code

“phlpwcspweb3”  is found at the “Amazon Web Services – Tagging Best Practices

From what I can decode from “phlpwcspweb3”  this is something related to web, and probably there are at least 3 instances of that kind.

According to AWS this should be meaningful hostname.

If you have decoded this you probably do not need to read further….

Continue reading

Running chef test-kitchen with newer vagrant

If you get

VBoxManage: error: The specified string / bytes buffer was to small. Specify a larger one and retry. (VERR_CFGM_NOT_ENOUGH_SPACE)

This means that the virtual machine needs a shorter name for the VM.

---
driver:
  name: vagrant
  customize:
    name: pdcsmb
    memory: 4144
    cpus: 4
  network:
    # - ["forwarded_port", {guest: 3000, host: 3000}]
    # - ["private_network", {ip: "192.168.33.33"}]

Bonus tip:

If you get

No live threads left. Deadlock?

Then remove all gems and reinstall chefdk.

Running chef kitchen with cookbook dependencies

Berksfile

source 'https://supermarket.chef.io'

metadata

group :test do
  cookbook 'company_firewall', path: "../company_firewall"
  cookbook 'company_nginx', path: "../company_nginx"
  cookbook 'company', path: "../company"
  cookbook 'data-incoming', path: "../data-incoming"
  cookbook 'aws', path: "../aws"
end

metadata.rb

name             'data-listener'
maintainer       'Company'
maintainer_email '[email protected]'
license          'All rights reserved'
description      'Installs/Configures data-listener'
long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
version          '0.2.32'

depends 'sysctl', '~> 1.0.5'
depends 'copany_firewall'
depends 'company_nginx'
depends 'aws'

 

mdadm: Duplicate MD device names in conf file where found.

remove the doublicate file from /etc/mdadm/mdadm.conf

cp mdadm.conf mdadm.conf.backup
grep -v "2 UUIDsomehing..." mdadm.conf.backup > mdadm.conf

Make sure the raid is up

mdadm --assemble --scan

Load the lvm volumes

lvm vgchange -ay
  • -a – activate
  • -y – complete

exit to try the changes

$(initramfs) exit

Now you system will boot

Making the changes permanent

Fix the /etc/mdadm/mdadm.conf file by removing the duplicate definition.

update-initramfs  -u -k all

If you are still in initramfs busybox

mkdir /target
mount /dev/md1 /target
mount -o bind /dev /target/dev
mount -o bind /proc /target/proc
mount -o bind /sys /target/sys
mount -o bind /dev/pts /target/dev/pts
chroot /target /bin/bash

Cheat

mdadm –detail /dev/md0 mdadm –detail /dev/md1

mdadm –detail –scan > /etc/mdadm/mdadm.conf

Stop an array & assemble array mdadm –stop /dev/md1

mdadm –assemble /dev/md0 /dev/sda1 /dev/sdb1 /dev/sdc1 /dev/sdd1 mdadm –assemble /dev/md1 /dev/sda2 /dev/sdb2 /dev/sdc2 /dev/sdd2

 

38693387_2127079967530580_4018998239729549312_n (1)

Building docker image with host network aka –net=host

There is no option to provide the net as param while building the docker images.
So the container doesn’t have internet from inside.
and is common to get this kind of errors

Running in NNNNNNNNN
Err http://http.debian.net wheezy Release.gpg
  Could not resolve 'http.debian.net'

One (stupid) solution is to put in the Docker file

RUN echo "nameserver 8.8.8.8" > /etc/resolv.conf; cat /etc/resolv.conf

Another is to help Docker with the networking.

Here is a solution is pointed here

# Forward chain between docker0 and eth0
iptables -A FORWARD -i docker0 -o eth0 -j ACCEPT
iptables -A FORWARD -i eth0 -o docker0 -j ACCEPT
# IPv6 chain if needed
ip6tables -A FORWARD -i docker0 -o eth0 -j ACCEPT
ip6tables -A FORWARD -i eth0 -o docker0 -j ACCEPT

And also to configure docker to use a dns server as pointed here

Docker official doc gives instruments to Configure a DNS server for use by Docker

Open the /etc/default/docker file for editing.

$ sudo nano /etc/default/docker

Add a setting for Docker.

DOCKER_OPTS="--dns 8.8.8.8"

Replace 8.8.8.8 with a local DNS server such as 192.168.1.1. You can also specify multiple DNS servers. Separated them with spaces, for example:

–dns 8.8.8.8 –dns 192.168.1.1

Warning: If you’re doing this on a laptop which connects to various networks, make sure to choose a public DNS server.

ps: nm-tool can be used to check local host DNS server

Save and close the file.

Restart the Docker daemon.

$ sudo restart docker

Replacement for tilda guake or other not working solutions

This script will activate gnome-terminal when you press INSERT (the key right to backspace) and will hide it when pressed again.

Using this script is better than tilda or guake or whatever because you can resize the terminal. You can clean the terminal and have all goods you have in normal terminal.

To install it under lubuntu/lxde open ~/config/openbox/lubuntu-rc.xml and add

https://gist.github.com/gudata/4065ad9bb1fd6d82933b

Here is the code for the riseterminal.sh

#/bin/bash
XDO=/usr/bin/xdotool

#TERMINAL_ID=`$XDO  search --name Terminal | /usr/bin/tail -1`

raise_or_fall () {
  TERMINAL_ID=$($XDO search --class gnome-terminal | /usr/bin/tail -1)
  CURRENT_WINDOW_ID=`$XDO getwindowfocus`

  WIDS=`$XDO search --onlyvisible --class gnome-terminal`

  if [ "$CURRENT_WINDOW_ID" -eq "$WIDS" ]
  then
    # zenity --question --text="down"
    $XDO windowminimize $TERMINAL_ID
  else
    # zenity --question --text="up"
    # $XDO windowraise $TERMINAL_ID
    # $XDO windowfocus --sync $TERMINAL_ID
     $XDO windowactivate $TERMINAL_ID
  fi
}

if `ps aux | grep "[g]nome-terminal" > /dev/null`
then
  raise_or_fall
else
  gnome-terminal &
fi

uptimed and uprecords

Nice! As I am maniac of  uptime here is nice utility found at http://www.linuxjournal.com/content/tracking-server-uptimes

uptimed. It is similar to the utility that most of us have heard of, uptime, except that it runs as a daemon and logs the system’s uptime instead of just reading info that is lost on a reboot. Uptimed provides a secondary command called uprecords

Autoicrement the serial in bind zone files

Argh. 30+ zone files and I have to increment all numbers by hand…no way.

Usage:  ./inc.rb in the current directory with all db files.

#!/usr/bin/ruby

Dir.glob("*.db") do |file_name|
 new_zone = [] Continue reading

How to access a server via sftp with ssh tunnel

Here is what we want to do

We have the following case:

Developer workstation windows with putty

Spider.bg – a host which has access to the www.psspy.se server

www.psspy.se – firewalled host – we want to put files here via sftp

What we want to do is to use spider.bg host as a proxy and connect to the psspy.se machine.

Tools that are used –

Here are the screenshots with the steps ….

Continue reading

© 2020 Gudasoft

Theme by Anders NorénUp ↑