Today I Learned

Problem

Transfering files in the background via rsync is possible via nohup command and some other workarounds.

These don't really work when your transfer takes days (>50TB of data) because the background job might get killed by the system.

In order to create a persistent session we need to use tmux.

Solution

Authentication

Create a ssh key and upload it to the server you want to transfer from.

ssh-keygen -t ed25519
ssh-copy-id -i ~/.ssh/id_rsa.pub [your_old_server.domain.com]

Create a new session and run rsync

tmux new -s [your_session_name]

rsync --avhPW --stats [your_old_server.domain.com]:[/source/path/] [/destination/path]

Detach from session

Press CTRL+B and then D. You session will detach and run in background.

Re-attach session

To get back into your session just:

tmux a -t [your_session_name]

That's it.

In order to use TouchID on MacBook Pro for Terminal sudo prompts, we need to enable Apple's Touch ID PAM module pam_tid.so (https://opensource.apple.com/source/pam_modules/pam_modules-173.1.1/modules/pam_tid/pam_tid.c.auto.html).

Just edit /etc/pam.d/sudo and add

auth sufficient pam_tid.so

Heads up, when you do a system update this change will be most probably overwritten. In order to make it persistent, you need to create a launchd daemon.

Create a new file called pam-tid.sh in a shared path

vim /Users/Shared/pam-tid.sh

#!/bin/bash

if ! grep 'pam_tid.so' /etc/pam.d/sudo --silent; then
  sed -i -e '1s;^;auth       sufficient     pam_tid.so\n;' /etc/pam.d/sudo
fi

Create a new com.graffino.pam.plist file:

vim /Users/Shared/com.graffino.pam.plist`

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.graffino.pam</string>

    <key>KeepAlive</key>
    <false/>

    <key>LaunchOnlyOnce</key>
    <true/>

    <key>RunAtLoad</key>
    <true/>

    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/pam-tid.sh</string>
    </array>
</dict>
</plist>

Start the daemon

Step 1. Make cron persist on restarts

unifi-os shell
curl -L https://github.com/unifi-utilities/unifios-utilities/raw/main/on-boot-script/packages/udm-boot_1.0.5_all.deb -o udm-boot.deb
dpkg -i udm-boot.deb
rm udm-boot.deb
exit

Step 2. Add root ssh keys on restart

cd /mnt/data/on_boot.d
vi 15-add-root-ssh-key.sh

File contents

#!/bin/sh

#####################################################
# ADD RSA KEYS AS BELOW - CHANGE BEFORE RUNNING     #
#####################################################
# set -- "ssh-rsa first key here all keys quoted" \ #
#        "ssh-rsa each line appended with slash " \ #
# 	 "ssh-rsa last one has no backslash"        #
#####################################################
set -- "ssh-rsa ..." \
        "ssh-rsa ...."

KEYS_FILE="/root/.ssh/authorized_keys"

counter=0
for key in "$@"
do
	# Places public key in ~/.ssh/authorized_keys if not present
	if ! grep -Fxq "$key" "$KEYS_FILE"; then
		let counter++
		echo "$key" >> "$KEYS_FILE"
	fi
done

echo $counter keys added to $KEYS_FILE

Make file executable and run it

chmod +x 15-add-root-ssh-key.sh 
./15-add-root-ssh-key.sh 

Step 3. Update banner

cat /dev/null > /issue
cat /dev/null > /etc/issue
cat /dev/null > /etc/motd

vi /etc/motd

# Insert your own banner 

Step 4. Update ssh configuration

UDM uses dropbear as ssh server and therefore the configuration is done on init.

Edit the dropbear configuration file

vi /etc/default/dropbear 
 
// See https://wiki.gentoo.org/wiki/Dropbear
DROPBEAR_OPTS="-sg"

Restart the dropbear service

/etc/init.d/dropbear restart

Sometimes you may need to add a little bit of javascript inside a Laravel Blade template. The proper way to do that is to use Laravel's @stack directive:

First, you need to add a @stack on the parent page or the layout:

<script src="{{ asset('js/app.js') }}"></script>
@stack('other-scripts')

Then you need to @push the respective script on the page you need it on.

@push('other-scripts')
<script>
  console.log('do something in js')
</script>
@endpush

The problem:

<?php if ( have_rows( 'repeater_field' ) ) while ( have_rows( 'repeater_field' ) ) : the_row(); ?>
    <?php
        $name = get_sub_field( 'name' );
        $age = get_sub_field( 'age' );
    ?>
	
    <p><?php echo $name; ?></p>
    <p><?php echo $age; ?></p>
<?php endwhile; ?>

This only gets you the normal, ascending order for those repeater rows. But what if you need to reverse the order of those rows, making first the last and vice-versa?

The Advanced Custom Fields documentation gives an example, but that didn't work at the time of this writing. They instruct you to use the get_field() function for the main repeater field, but for some reason that returns null on a repeater field.

What I've found to work was to use the get_sub_field() to get the field and the krsort() php method which sorts an array by key in descending order.

<?php
    $repeater = get_sub_field('repeater_field');
    krsort($repeater);
?>
<?php foreach ($repeater as $row): ?>
    <p><?php echo $row['name']; ?></p>
    <p><?php echo $row['age']; ?></p>
<?php endforeach; ?>

One thing to note is that with this method you need to change the way you access those variables.

Whenever you try to athenticate macOS asks for a PIN instead of a password. This happens if you paired your key at some point with macOS.

Just open Terminal.app and unpair it:

sudo /usr/sbin/sc_auth unpair -u YourUserName

or simply

sudo /usr/sbin/sc_auth unpair

Take out your key if you have it plugged in and reboot.

Your key should be unpaired from your username. Remember you don't have to pair your key to use it. You only have to pair it if you want to use it for macOS authentication.

The filter: blur() property in css may come in handy in some situations, but you should know of a side effect it has in Safari (tested on version 15.1 as of this writing).

The issue is a nasty performance drop around the element on which the filter is used, making every other interaction extremely slow. And it happens in Safari only. In other browsers everything works well.

One solution I've found so far is to use transform: translate3d(0, 0, 0) along with filter on the same element, to force the browser use GPU acceleration for that particular element instead of the CPU.

filter: blur(200px);
transform: translate3d(0, 0, 0);

Here are 2 ways to measure the performance of a function in javascript:

console.time('function');
function();
console.timeEnd('function');

or:

const startTime = performance.now();
function();
const endTime = performance.now();
console.log(`Function took ${endTime - startTime} milliseconds`);

To automatically update your global NodeJS packages just issue the following command. It will list outdated packages and update them via npm install -g.

npm install -g $(npm outdated -g | cut -d' ' -f 1 | sed '1d' | xargs -I '$' echo '$@latest' | xargs echo)

The default src attribute acts as fallback for a default image size for browsers that have no support for srcset:

<img src="/images/image.jpg">

The srcset attribute gives the browser a bunch of different sizes of the same image to choose from.

The name of the image doesn't matter, you could name them image-small, image-medium and so on, but the 375w part should match the real width of the image:

<img
    src="/images/image.jpg"
    srcset="/images/image-375.jpg 375w,
            /images/image-768.jpg 768w,
            /images/image-1440.jpg 1440w,
            /images/image-1920.jpg 1920w,
            /images/image-2560.jpg 2560w"
>

Adding the sizes attribute specifies what image size to choose at a specific breakpoint and above (min-width) or below (max-width) it:

<img
    src="/images/image.jpg"
    srcset="/images/image-375.jpg 375w,
            /images/image-768.jpg 768w,
            /images/image-1440.jpg 1440w,
            /images/image-1920.jpg 1920w,
            /images/image-2560.jpg 2560w"
    sizes="(min-width: 1920px) 1400px, (min-width: 1600px) 1000px, (min-width: 768px) 700px, 300px"
>

One thing to note here is that the browser will first calculate the device width and then look at the sizes attribute and choose the first matching condition.

If you set for example sizes="(max-width: 1920px) 1500px, (max-width: 768px) 600px, 300px", and the device width is 640px, the first matching condition will be max-width: 1920px, therefore it will choose the 1500px image width suggestion, rather than the 600 you most probably expected. The same can happen with the min-width if the order is not right.

Notice also that we are asking the browser to choose a 1000px wide image between 1600px and 1919px, but in the srcset sources, there's no image at 1000px. Here, the browser will choose the image with the exact size, if there is one, or the closest one if not, but higher in size than specified, in this case, 1000px. Therefore, the 1440px will be selected.

It can take a while to grasp how this works, but it's worth taking the time to understand it.

When someone deletes a git tag on a remote and creates a new one with the same name, you might get a similar error like the one below while trying to do a git command, a git fetch for example.

![rejected]           2.10.14    -> 2.10.14 (would clobber existing tag)

The solution is to update your local tags with the new remote tags. This is how:

git fetch --tags -f

We're heavily using AWS and we're scripting everything we can. As AWS CLI 2 came out recently we needed to update our scripts.

This script creates a new snapshot and deletes all snapshots older than 2 weeks, for a specific volume.

create-snapshot-and-cleanup.sh

#!/bin/bash

DESCRIPTION="example.com"
VOLUME="vol-xxxxxxxxxx"

SNAPSHOT_AGE=$(date +%Y-%m-%d --date '2 weeks ago')
TODAY=$(date +%d-%m-%Y)

echo "Creating new snapshot of volume $VOLUME."
aws ec2 create-snapshot --output text --description "$DESCRIPTION - AutoSnapshot $TODAY" --volume-id $VOLUME >> /dev/null

echo "Deleting snapshots older than: $SNAPSHOT_AGE"

snapshots=$(aws ec2 describe-snapshots --output text --filters Name=volume-id,Values=$VOLUME --query "Snapshots[?StartTime<'$SNAPSHOT_AGE'].SnapshotId")

echo "Snapshots sheduled for deletion: $snapshots"

for snapshot in $snapshots; do
  echo "Deleting $snapshot ..."
  aws ec2 delete-snapshot --snapshot-id $snapshot
done

If you ever need to check what php version do composer packages and their dependencies require, there's a special command just for that.

composer check-platform-reqs

The output should look something like below. Maybe without the failed steps.

ext-dom        20031129                                                success  
ext-fileinfo    7.3.19                                                  success  
ext-filter      7.3.19                                                  success  
ext-json       1.7.0                                                   success  
ext-libxml     7.3.19                                                  success  
ext-mbstring   7.3.19                                                  success  
ext-openssl    7.3.19                                                  success  
ext-pcre       7.3.19                                                  success  
ext-Phar       7.3.19                                                  success  
ext-SimpleXML  7.3.19                                                  success  
ext-tokenizer  7.3.19                                                  success  
ext-xml        7.3.19                                                  success  
ext-xmlwriter  7.3.19                                                  success  
lib-pcre       10.32                                                   success  
php            7.3.19    league/commonmark requires php (^7.4 || ^8.0) failed   
php            7.3.19    league/config requires php (^7.4 || ^8.0)      failed   

This is useful when your app fails due to unfulfilled dependency requirements.

Managing multiple hosts is a pain when using SSH key authentication. There are a lot of solutions out there for managing SSH keys, there's Ansible, Puppet, or other paid solutions.

There's also another option, to use a script to do the job for you.

Here's how

1. Create a targets file containing your hosts and usernames

# Host
hostname.example.com username

2. Create a add-keys.sh file

#!/bin/bash
keys=${1:-*.pub}
echo update ssh-keys: ${keys}

cat targets | grep -vE '^(\s*$|#)' | sed 's/#.*$//g' | while read host user
do
   echo "# Adding public ssh-keys for $user@$host"
   for k in ${keys};
   do
      echo "# Adding public key $k"
      touch ${k%\.pub}
      ssh-copy-id -f -i $k $user@$host
   done
done

3. Create a remove-keys.sh file

#!/bin/bash

keys=${1:-*.pub}
echo update ssh-keys: ${keys}

cat targets | grep -vE '^(\s*$|#)' | sed 's/#.*$//g' | while read host user
do
   echo "# Remove public ssh-keys for $user@$host"
   for k in ${keys};
   do
      echo "# Remove public key $k"
      key=$(<$k)
      ssh $user@$host 'bash -s' <<EOT
sed -i "/$key/d" ~/.ssh/authorized_keys
EOT
      echo "# Key removed"
   done
done

4. Add the public keys you want to add as .pub files in the same folder

5. Run one of the scripts. Done.

Have you ever been in a situation where you're a cog in a big company your work depends on other people's action and you can't get them to do their part so that you can finish your own tasks?

If the answer is yes, you know how frustrating this can be. This happens because the other people involved have their own priorities and you're at the very bottom.

In order to get things done, you have to get to the top of their list and the solution is rather simple:

Create a carefully non-personal, non-emotional email in which you explain the situation, place the person responsibile in the "To:" field and add their direct manager, your own manager, and anyone remotely responsible in the "Cc" field.

I guarantee your problem will be solved in no time. :D

You can even escalate this further up the chain with a follow-up email where you add more managers up the chain in the "Cc:" field.

Don't forget to thank everyone involved after your issue is solved.