Moving lvm-thin volumes on proxmox between vm-s or ct-s

| Leave a comment

it’s pretty straight forward following this official howto: https://pve.proxmox.com/wiki/Moving_disk_image_from_one_KVM_machine_to_another

lvs shows you all volumes in their volume group (in my case ‘ssd’)

LV               VG  Attr       LSize    Pool   Origin Data%  Meta%  Move Log Cpy%Sync Convert
data             pve twi-a-tz--   32.12g               0.00   1.58                            
root             pve -wi-ao----   16.75g                                                      
swap             pve -wi-ao----    8.00g                                                      
guests           ssd twi-aotz--   <2.33t               74.93  45.51                           
vm-100-disk-0    ssd Vwi-a-tz--   12.00g guests        72.69                                  
vm-101-disk-0    ssd Vwi-a-tz--   12.00g guests        85.22                                  
vm-101-disk-1    ssd Vwi-a-tz--   50.00g guests        99.95                                  
vm-102-disk-0    ssd Vwi-a-tz--   12.00g guests        97.57                                  
vm-102-disk-1    ssd Vwi-a-tz--   50.00g guests        64.54                                  
vm-103-disk-0    ssd Vwi-a-tz--   12.00g guests        74.37                                  
vm-103-disk-1    ssd Vwi-a-tz--  150.00g guests        52.42                                  
vm-104-disk-0    ssd Vwi-a-tz--   12.00g guests        90.74                                  
vm-104-disk-1    ssd Vwi-a-tz--   10.00g guests        95.27                                  
vm-105-disk-0    ssd Vwi-a-tz--   12.00g guests        55.79                                  
vm-105-disk-1    ssd Vwi-a-tz--   10.00g guests        32.89                                  
vm-106-disk-0    ssd Vwi-a-tz--   12.00g guests        77.78                                  
vm-106-disk-1    ssd Vwi-a-tz--   10.00g guests        99.82                                  
vm-107-disk-0    ssd Vwi-a-tz--   32.00g guests        0.00                                   
vm-107-disk-1    ssd Vwi-a-tz--  500.00g guests        95.41                                  
vm-108-disk-0    ssd Vwi-aotz--    8.00g guests        43.73                                  
vm-109-disk-0    ssd Vwi-a-tz--   12.00g guests        52.41                                  
vm-109-disk-1    ssd Vwi-a-tz--   50.00g guests        2.22                                   
vm-110-disk-0    ssd Vwi-a-tz--   12.00g guests        51.14                                  
vm-110-disk-1    ssd Vwi-a-tz--   50.00g guests        2.22                                   
vm-111-disk-0    ssd Vwi-a-tz--   12.00g guests        84.85                                  
vm-111-disk-1    ssd Vwi-a-tz--  100.00g guests        16.97                                  
vm-112-disk-0    ssd Vwi-a-tz--    8.00g guests        13.53                                  
vm-113-disk-0    ssd Vwi-a-tz--    8.00g guests        11.55                                  
vm-114-disk-0    ssd Vwi-a-tz--   16.00g guests        84.31                                  
vm-115-disk-0    ssd Vwi-a-tz--   16.00g guests        97.12                                  
vm-116-disk-0    ssd Vwi-a-tz--    8.00g guests        31.49                                  
vm-117-cloudinit ssd Vwi-aotz--    4.00m guests        50.00                                  
vm-117-disk-0    ssd Vwi-aotz--   10.00g guests        39.71                                  
vm-117-disk-1    ssd Vwi-aotz-- 1000.00g guests        97.47

If the id of the new ct or vm is not equal to the id of the volume’s previous attachment, rename them, i.e.

lvrename ssd/vm-101-disk-1 ssd/vm-117-disk-2

this will make vm-101-disk-1 available as vm-117-disk-2, you have to increase the count in the end of the name.

then edit the config of the actual vm.

take the line from /etc/pve/qemu-server/<vm id>.conf that describes the volume to the new <vm id>.conf

the tricky thing was to run qm rescan afterwards which fixed syntax and made the volume appear in the web gui where i could finally attache it to the new vm.

WakeOnLan, Archlinux, systemd-networkd, Asus Pro WS X570-ACE

| Leave a comment

The board has two integrated ethernet adapters, here’s the lshw data:

sudo lshw -c network 
  *-network                  
       description: Ethernet interface 
       product: I211 Gigabit Network Connection 
       vendor: Intel Corporation 
       physical id: 0 
       bus info: pci@0000:05:00.0 
       logical name: enp5s0 
       version: 03 
       serial: 24:4b:fe:<redacted>
       size: 1Gbit/s 
       capacity: 1Gbit/s 
       width: 32 bits 
       clock: 33MHz 
       capabilities: pm msi msix pciexpress bus_master cap_list ethernet physical tp 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation 
       configuration: autonegotiation=on broadcast=yes driver=igb driverversion=5.12.8-zen1-1-zen duplex=full firmware=0. 6-1 ip=<redacted> latency=0 link=yes multicast=yes port=twisted pair speed=1Gbit/s 
       resources: irq:61 memory:fc900000-fc91ffff ioport:e000(size=32) memory:fc920000-fc923fff 
  *-network 
       description: Ethernet interface 
       product: RTL8111/8168/8411 PCI Express Gigabit Ethernet Controller 
       vendor: Realtek Semiconductor Co., Ltd. 
       physical id: 0.1 
       bus info: pci@0000:06:00.1 
       logical name: enp6s0f1 
       version: 1a 
       serial: 24:4b:fe:<redacted>
       size: 1Gbit/s 
       capacity: 1Gbit/s 
       width: 64 bits 
       clock: 33MHz 
       capabilities: pm msi pciexpress msix bus_master cap_list ethernet physical tp mii 10bt 10bt-fd 100bt 100bt-fd 1000bt-fd autonegotiation 
       configuration: autonegotiation=on broadcast=yes driver=r8169 driverversion=5.12.8-zen1-1-zen duplex=full firmware=rtl8168fp-3_0.0.1 11/16/19 ip=<redacted> latency=0 link=yes multicast=yes port=twisted pair speed=1Gbit/s 
       resources: irq:24 ioport:d800(size=256) memory:fc814000-fc814fff memory:fc808000-fc80bfff

It seems that the UEFI entry to activate Wake on Lan for PCIe devices only affects the Intel port, i’ve persistently activated WOL for the realtek port by adding a .link file to /etc/systemd/network/foobar.link

[Match] 
MACAddress=<redacted>
 
[Link] 
WakeOnLan=magic
# below lines are cloned from original entry in
# /usr/lib/systemd/network/99-default.link
# which is the default link file for all adapters whose section is hereby overwritten
NamePolicy=keep kernel database onboard slot path 
AlternativeNamesPolicy=database onboard slot path 
MACAddressPolicy=persistent
 
The arch wiki shows a couple of alternative ways, but this seems to be the most straight forward for me.

Upgrade Postgresql from 11 upwards

| Leave a comment

On Ubuntu 18.04

Multiple installations (11, 12, 13) be wary of that, as pg_upgradcluster for example will always go for the hightes version.

copied configuration files for new version

cp -R  /etc/posgresql/11 /etc/posgresql/12

initialized new version db

/usr/lib/postgresql/12/bin/initdb -D /srv/postgres/12/main

stopped the current server and killed all connections

/usr/lib/postgresql/11/bin/pg_ctl -D /srv/postgres/11/main/ -mf stop

ran checked upgrade with linked files

time /usr/lib/postgresql/12/bin/pg_upgrade --old-bindir /usr/lib/postgresql/11/bin/ --new-bindir /usr/lib/postgresql/12/bin/ --old-datadir /srv/postgres/11/main/ --new-datadir /srv/postgres/12/main/ --link --check

had to fix diverse configuration file problems that are obvious when running 

"/usr/lib/postgresql/11/bin/pg_ctl" -w -l "pg_upgrade_server.log" -D "/srv/postgres/11/main" -o "-p 50432 -b  -c listen_addresses='' -c unix_socket_permissions=0700 -c unix_socket_directories='/var/lib/postgresql'" start
cat pg_upgrade_server.log

mostly faulty references to configuration files, or having to make explicit the non-standard data dir location.

then the systemd related things

systemctl disable postgres@11-main
systemctl enable postgres@12-main

This place was most helpful:
https://blog.crunchydata.com/blog/how-to-perform-a-major-version-upgrade-using-pg_upgrade-in-postgresql

Some reminders for http caching

| Leave a comment

blatantly copypasted from https://httptoolkit.tech/blog/http-wtf/

No-cache means “do cache”

Caching has never been easy, but HTTP cache headers can be particularly confusing. The worst examples of this are no-cache and private. What does the below response header do?

Cache-Control: private, no-cache

It looks like this means “don’t store this response anywhere”, right?

Hahaha no.

In reality, this means “please store this response in all browser caches, but revalidate it when using it”. In fact, this makes responses more cacheable, because this applies even to responses that wouldn’t normally be cacheable by default.

Specifically, no-cache means that your content is explicitly cacheable, but whenever a browser or CDN wants to use it, they should send a request using If-Match or If-Modified-Since to ask the server whether the cache is still up to date first. Meanwhile private means that this content is cacheable, but only in end-client browsers, not CDNs or proxies.

If you were trying to disable caching because the response contains security or privacy sensitive data that shouldn’t be stored elsewhere, you’re now in big trouble. In reality, you probably wanted no-store.

If you send a response including a Cache-Control: no-store header, nobody will ever cache the response, and it’ll come fresh from the server every time. The only edge case is if you send that when a client already has a cached response, which this won’t remove. If you want to do that and clear existing caches too, add max-age=0.

Twitter notably hit this issue. They used Pragma: no-cache (a legacy version of the same header) when they should have used Cache-Control: no-store, and accidentally persisted every user’s private direct messages in their browser caches. That’s not a big problem on your own computer, but if you share a computer or you use Twitter on a public computer somewhere, you’ve now left all your private messages conveniently unencrypted & readable on the hard drive. Oops.

SpinRite 6 on external Toshiba usb disk

| Leave a comment

AFter 827 days of running time my RaspiBlitz BTC lightning node refused to mount the external hdd (Toshiba HDTB410EK3AA Canvio Basics, USB 3.0, 1TB). Smart errors of the weirdest kind. I remembered Gibson’s spammy advertisements during the Security Now! Podcast, praising SpinRite for recovery. As there was no physical damage / interaction that would have caused that i gave it a try.

After i bought the license, i downloaded the exe causing first problem, how to run on Linux? I have a Windows 7 laptop for such cases, so i executed the program and tried all the different options to create a bootable USB, finally succeeding by writing out the diskette spinrite.img to harddisk, then dd-ing it onto a usb flash drive:

dd if=/path/to/SpinRite.img conv=notrunc of=/dev/<your usb device, i.e. sda>

After rebooting the same laptop with the external USB disk attached, SpinRite started right away, and luckily for me, the drive was instantly recognized; no need for driver voodoo on the included FreeDOS distribution – that was my biggest concern. Probably the fact that the external disk is not a casing with some exotic usb-controller, but a disk with an integrated usb port helped a lot. A small downer was the unavailability of smart data for SpinRite – I don’t have a theory about that.

The first run failed with a program abort:

This is ongoing.

run openvpn in client mode automatically after linux boot

| Leave a comment

scenario: send out a raspberry pi model b rev1, all setup with raspberryi os / raspbian.

the hardware specs are nothing much, but the machine is reliable, even when apparently half the ram chips are dead….

install openvpn, then take the config file from the server you want to connect to – in my case an ovpn file generated by pivpn – and put it into the config folder `/etc/openvpn/`. if your vpn profile is password protected, just add a simple textfile with the cleartext pass and reference it in your vpn profile file like so:
askpass /etc/openvpn/passwordfilename

make sure openvpn.service is started and enabled.
systemctl enable openvpn && systemctl restart openvpn

should be it, ip a should show you the tunnel interface already.

ps: for the routing, make sure that your that your router has a static entry that sends all the traffic to the vpn subnet to the vpn server, but that is something that depends really on your own net topology.

update gnubee debian jessie to buster

| Leave a comment

thanx to https://feeding.cloud.geek.nz/posts/installing-debian-buster-on-gnubee2/

Upgrade to stretch and then buster

To upgrade to stretch, put this in /etc/apt/sources.list:

deb http://httpredir.debian.org/debian stretch main
deb http://httpredir.debian.org/debian stretch-updates main
deb http://security.debian.org/ stretch/updates main

Then upgrade the packages:

apt update
apt full-upgrade
apt autoremove
reboot

To upgrade to buster, put this in /etc/apt/sources.list:

deb http://httpredir.debian.org/debian buster main
deb http://httpredir.debian.org/debian buster-updates main
deb http://security.debian.org/debian-security buster/updates main

and upgrade the packages:

apt update
apt full-upgrade
apt autoremove
reboot

Configure Ubuntu 18.04 with grub2 to activate serial console

| Leave a comment

Thanks to hiroom2

1 /etc/default/grub

  • Change GRUB terminal to console and ttyS0. This will provide one GRUB to a monitor display and serial console.
  • Change linux kernel console to tty1 and ttyS0. This setting will be taken over to userland, and there will be two login prompt for tty1 and ttyS0. 
GRUB_CMDLINE_LINUX="console=tty1 console=ttyS0,115200"
GRUB_TERMINAL="console serial"
GRUB_SERIAL_COMMAND="serial --speed=115200 --unit=0 --word=8 --parity=no --stop=1"