Compare two filesystems

Lets run those command on the machines

New instanceOld instance
find / -xdev | sort > new.txt find / -xdev | sort > old.txt

Pull the files locally

scp -i ~/.ssh/somekey [email protected]:/new.txt  /tmp/new.txt
scp -i ~/.ssh/somekey [email protected]:/old.txt  /tmp/old.txt

Then use this great delta tool to compare the files

delta -s /tmp/new.txt /tmp/old.txt

How to pull files from remote sftp

Create a script with the commands to pull the files and the remove them from the remote server

get -r upload/* incoming/
rm upload/*

You will need a cron

0 5 * * * /usr/bin/sftp -b batchfile.sh [email protected]

I recommend using systemd so that you can have logs

Where is more clear to do line break

For decades the recommended style was to break after binary operators. But this can hurt readability in two ways: the operators tend to get scattered across different columns on the screen, and each operator has moved away from its operand and onto the previous line. Here, the eye has to do extra work to tell which items are added and which are subtracted:

income = (gross_wages +
          taxable_interest +
          (dividends - qualified_dividends) -
          ira_deduction -
          student_loan_interest)

To solve this readability problem, mathematicians and their publishers follow the opposite convention. Donald Knuth explains the traditional rule in his Computers and Typesetting series: “Although formulas within a paragraph always break after binary operations and relations, displayed formulas always break before binary operations” [1]

Following the tradition from mathematics usually results in more readable code:

# easy to match operators with operands
income = (gross_wages
          + taxable_interest
          + (dividends - qualified_dividends)
          - ira_deduction
          - student_loan_interest)

Source:

https://www.python.org/dev/peps/pep-0008/#should-a-line-break-before-or-after-a-binary-operator

[1] Donald Knuth’s The TeXBook, pages 195 and 196.

Create AP on ubuntu

This is the most important taken from this great post

Checks

iw list | grep "Supported interface modes" -A 8
Supported interface modes:
* IBSS
* managed
* <strong>AP</strong>
* monitor
* mesh point
* P2P-client
* P2P-GO
* P2P-device

Search for AP

Install

sudo apt-get update && sudo apt-get install hostapd

The /etc/hostapd/hostapd.conf is the main configuration which you need to deal with in order to set up a SoftAP.

This is the minimal configuration setting which will let you test if hostapd is working. Create a file ~/hostapd-test.conf
with the following content:

#change wlan0 to your wireless device
interface=wlan0
driver=nl80211
ssid=test
channel=1

The driver is nl80211, don’t try to change it.

Start hostapd by

sudo hostapd ~/hostapd-test.conf

You will be able to connect but without any Internet. Just check you see the test wifi.
Once hostapd is working, its time to configure hostapd with more options.

#sets the wifi interface to use
interface=wlan0
#driver to use, nl80211 works in most cases
driver=nl80211
#sets the ssid of the virtual wifi access point
ssid=myhotspot
#sets the mode of wifi, depends upon the devices you will be using. It can be a,b,g,n. Not all cards support 'n'.
hw_mode=g
#sets the channel for your wifi
channel=6
#macaddr_acl sets options for mac address filtering. 0 means "accept unless in deny list"
macaddr_acl=0
#setting ignore_broadcast_ssid to 1 will disable the broadcasting of ssid
ignore_broadcast_ssid=0
#Sets authentication algorithm
#1 - only open system authentication
#2 - both open system authentication and shared key authentication
auth_algs=1
#####Sets WPA and WPA2 authentication (remove this section if you don't need encryption)#####
#wpa option sets which wpa implementation to use
#1 - wpa only
#2 - wpa2 only
#3 - both
wpa=3
#sets wpa passphrase required by the clients to authenticate themselves on the network
wpa_passphrase=KeePGuessinG
#sets wpa key management
wpa_key_mgmt=WPA-PSK
#sets encryption used by WPA
wpa_pairwise=TKIP
#sets encryption used by WPA2
rsn_pairwise=CCMP

complete /etc/hostapd/hostapd.conf with WPA authentication.

interface=wlan0
driver=nl80211
ssid=YOURHOTSOPTNAME
hw_mode=g
channel=6
macaddr_acl=0
auth_algs=1
ignore_broadcast_ssid=0
wpa=3
wpa_passphrase=THEPASSWORD
wpa_key_mgmt=WPA-PSK
wpa_pairwise=TKIP
rsn_pairwise=CCMP

Set IP for the adapter

# Setup the interface
$ ip link set wlp5s0 down
$ ip addr flush dev wlp5s0
$ ip link set wlp5s0 up
$ ip addr add 10.0.0.1/24 dev wlan0
# start hostapd
$ sudo killall dnsmasq; dnsmasq
$ sudo hostapd

SHARING THE INTERNET

Refer to https://wiki.archlinux.org/index.php/Internet_sharing for more information or how to achieve the same using nftables. Ignore this step if all you want to do is setup an internal wifi network and clients don’t need access to any external network.

########### Enable NAT ############
iptables -t nat -A POSTROUTING -o $2 -j MASQUERADE
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i $1 -o $2 -j ACCEPT
 
#Thanks to lorenzo
#Uncomment the line below if facing problems while sharing PPPoE, see lorenzo's comment for more details
#iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
 
sysctl -w net.ipv4.ip_forward=1

SETTING UP THE DHCP SERVER

You will be totally fine to go with manual IP settings…but… here are few options

dnsmasq

apt-get install dnsmasq

The default /etc/dnsmasq.conf explains all its configuration options pretty well, so I will jump straight to what a simple /etc/dnsmasq.conf should look like

# Interface to bind to
interface=wlan0
# Specify starting_range,end_range,lease_time
dhcp-range=10.0.0.3,10.0.0.20,12h
 
# Uncomment and modify the following lines if you don't want to forward dns from the host's /etc/resolv.conf
## disables dnsmasq reading any other files like /etc/resolv.conf for nameservers
#no-resolv
## dns addresses to send to the clients
#server=8.8.8.8
#server=8.8.4.4
# Setup the interface
$ ip link set wlp5s0 down
$ ip addr flush dev wlp5s0
$ ip link set wlp5s0 up
$ ip addr add 10.0.0.1/24 dev wlan0
# start hostapd
$ sudo killall dnsmasq; dnsmasq
$ sudo hostapd

dhcpd

/etc/default/isc-dhcp-server

INTERFACES="enp1s0"

cat /etc/dhcp/dhcpd.conf

# dhcpd.conf
#
# Sample configuration file for ISC dhcpd
#
# Attention: If /etc/ltsp/dhcpd.conf exists, that will be used as
# configuration file instead of this file.
#

# option definitions common to all supported networks...
#option domain-name "example.org";
#option domain-name-servers ns1.example.org, ns2.example.org;

default-lease-time 600;
max-lease-time 7200;

# The ddns-updates-style parameter controls whether or not the server will
# attempt to do a DNS update when a lease is confirmed. We default to the
# behavior of the version 2 packages ('none', since DHCP v2 didn't
# have support for DDNS.)
ddns-update-style none;

# If this DHCP server is the official DHCP server for the local
# network, the authoritative directive should be uncommented.
authoritative;

# Use this to send dhcp log messages to a different log file (you also
# have to hack syslog.conf to complete the redirection).
log-facility local7;

subnet 180.72.95.0 netmask 255.255.255.0 {
}

subnet 192.168.4.0 netmask 255.255.255.0 {
   range 192.168.4.100 192.168.4.254;
   option subnet-mask 255.255.255.0;
   option domain-name-servers 192.168.4.1;
   option routers 192.168.4.1;
}

sudo service  dhcpd.service restart

The final script

#!/bin/bash
# Usage: ./initSoftAP
########### Initial wifi interface configuration #############
ip link set $1 down
ip addr flush dev $1
ip link set $1 up
ip addr add 10.0.0.1/24 dev $1
 
# If you still use ifconfig for some reason, replace the above lines with the following
# ifconfig $1 up 10.0.0.1 netmask 255.255.255.0
sleep 2
###########
 
########### Start dnsmasq ##########
if [ -z "$(ps -e | grep dnsmasq)" ]
then
 dnsmasq
fi
###########
########### Enable NAT ############
iptables -t nat -A POSTROUTING -o $2 -j MASQUERADE
iptables -A FORWARD -m conntrack --ctstate RELATED,ESTABLISHED -j ACCEPT
iptables -A FORWARD -i $1 -o $2 -j ACCEPT
 
#Thanks to lorenzo
#Uncomment the line below if facing problems while sharing PPPoE, see lorenzo's comment for more details
#iptables -I FORWARD -p tcp --tcp-flags SYN,RST SYN -j TCPMSS --clamp-mss-to-pmtu
 
sysctl -w net.ipv4.ip_forward=1
###########
########## Start hostapd ###########
hostapd /etc/hostapd/hostapd.conf
killall dnsmasq

Check to dns zones

We are migrating from Cloudflare to route53 and want to make sure that all the records are migrated correctly.

records = %w(
  www
  blog
)

records.each do |record|
  aws = `dig +noall +answer www.example.com |tr '\t' ' ' | cut -d' ' -f1,3,4,5`
  cf = `dig +noall +answer www.example.com @1.1.1.1|tr '\t' ' ' | cut -d' ' -f1,3,4,5`

  if aws != cf
    puts("\nerror:")
    puts(aws)
    puts(cf)
  end
end

terraform bash shorcuts

Do you enjoy switching between us-east-1 and eu-central-1? If not here are some shortcuts which might make your day.

function change_region(){
  folder_name=`pwd`
  new_region=$1
  old_region=$2
  new_folder="${folder_name/$old_region/$new_region}"
  cd $new_folder
}

function production(){
  change_region 'terraform_production' 'terraform_development'
}


function development(){
  change_region 'terraform_development' 'terraform_production'
}

function us(){
  change_region 'us-east-1' 'eu-central-1'
}


function eu(){
  change_region 'eu-central-1' 'us-east-1'
}


How to give better examples

Giving an example with foo and bar is not a good idea because the brain needs to decode foo to something which is known what it is.

If you start with foo and bar, and the example is complicated in the end you will have in the head only foobars and nothing will be clear.

If you give the example with real-world names. Foo is Coffee, Bar is Cookie, then it is easy.

Here is another example from SQLAlchemy where they give an example for parent-chield relations with parent and chields.

For me, it is much clear if I work with some real-world objects like User has and belongs to many Alerts which has many Users and Subscriptions. Subscriptions have and belong to many users.

Many to Many

class Alert(Base):
tablename = 'alerts'
left_id = Column(Integer, ForeignKey('left.id'), primary_key=True)
right_id = Column(Integer, ForeignKey('right.id'), primary_key=True)
extra_data = Column(String(50))
subscription = relationship("Subscription", back_populates="users")
user = relationship("User", back_populates="subscriptions")
class User(Base):
tablename = 'left'
id = Column(Integer, primary_key=True)
subscriptions = relationship("Alert", back_populates="user")
class Subscription(Base):
tablename = 'right'
id = Column(Integer, primary_key=True)
users = relationship("Alert", back_populates="subscription")

Many to One

class Subscription(Base):
    __tablename__ = 'subscriptions'
    id = Column(Integer, primary_key=True)
    items = relationship("Item", back_populates="subscription")

class Item(Base):
    __tablename__ = 'items'
    id = Column(Integer, primary_key=True)
    subscription_id = Column(Integer, ForeignKey('subscriptions.id'))
    subscription = relationship("Subscription", back_populates="items")    

Rails free hosting

If you want free hosting, I mean free, not-cheap or other kinds of hosting. The only option which I know for now is Heroku.

There are some websites which advertise them as free rails hosting but they are paid.

So stay safe with the good old, slow on boot but very stable and enjoyable to work Heroku!

Akonadi KDE

I try to keep my system fast and to know what is running. I was surprised that today I found one mysql instance which was running without my explicit permission. This was the akonadi service.

Akonadi service is some kde tooling for keeping contacts, calendar and notes. Here is a full list of the apps

accountwizard akonadiconsole kalarm themeeditors kmail knotes konsolekalendar kontact korganizer ktnef

If you want to purge it from speed optimized linux do:

rm -r /home/guda/.local/share/akonadi
rm -rf /home/guda/.config/akonadi
rm -rf /home/guda/.kde/share/config/akonadi-firstrunrc

apt-get remove --purge akonadi-backend-mysql akonadi-backend-postgresql akonadi-backend-sqlite mariadb-server-core-10.3
pkill -f akona

Restore it back with sqlite database

I will try to run it for a while with sqlite database to see if there will be speed/ram problems

First change the database driver in settings to sqlite

mkdir /home/guda/.config/akonadi/
cat > /home/guda/.config/akonadi/akonadiserverrc
[%General]
Driver=QSQLITE3

[QSQLITE3]
Name=/home/guda/.local/share/akonadi/akonadi.db

[Debug]
Tracer=null

Then install some packages

apt install akonadi-backend-sqlite kdepim-runtime  kdepim
apt install kjots 

Then as YOURSELF run

akonadictl start

And if you want to see all the …apps using Akonadi run

akonadiconsole

Make tables visible on postgres after pgloader

Hello folks,

You have just used the pgloader to import some database in postgres but when you do

space_util=# \dt
Did not find any relations.

The problem is that you have to fix the search path for your tables.

Here is how to do it (or check the link for more ways)

space_util=# ALTER DATABASE space_util SET search_path = space_util,public;
space_util=# \dt
                                    List of relations
   Schema   |                           Name                           | Type  |  Owner   
------------+----------------------------------------------------------+-------+----------
 space_util | some_nice_table | table | postgres

© 2020 Gudasoft

Theme by Anders NorénUp ↑