diceline-chartmagnifiermouse-upquestion-marktwitter-whiteTwitter_Logo_Blue

Today I Learned

Fastest way to delete a folder structure in Linux

The problem

Deleting a huge folder structure is a pain and can take forever. However if you want to take forever divided by 3, you can use rsync.

It seems rsync is 3 time faster when deleting a huge folder structure than rm or find because it doesn't read the files before deleting.

The trick

  • Create an empty folder in /tmp/empty_folder
  • Run the command:

rsync -aP --delete ./empty_folder/ ./target_folder/


Enjoy!

HTTP pools - Laravel HTTP client

In PHP, unlike JavaScript, we don't have native support for Promises or asynchronous programming. This limitation can become apparent when dealing with multiple HTTP requests that need to be made simultaneously. However, the Http::pool method, as shown in the code snippet, offers a workaround by allowing us to leverage parallel processing.

The code uses the HTTP pools functionality provided by the Laravel HTTP client, which is a convenient way to execute multiple HTTP requests concurrently. Here's an expanded explanation of the code:

//dummy code

$responses = Http::pool(fn (Pool $pool) => [
    $pool->get("https://jsonplaceholder.typicode.com/todos/1"),
    $pool->get("https://jsonplaceholder.typicode.com/users/1"),
    $pool->get("https://jsonplaceholder.typicode.com/posts/1")
]);

$todoInfo = $responses[0]->json();
$userInfo = $responses[1]->json();
$postInfo = $responses[2]->json();

This code snippet demonstrates how to make three (or more) simultaneous HTTP calls to different endpoints using HTTP pools in PHP. In a specific situation on the project I'm working on, this approach significantly reduced the load time from 3 seconds to 1.1 seconds.

Transfer TB of files in the background via rsync and tmux on linux

Problem

Transfering files in the background via rsync is possible via nohup command and some other workarounds.

These don't really work when your transfer takes days (>50TB of data) because the background job might get killed by the system.

In order to create a persistent session we need to use tmux.

Solution

Authentication

Create a ssh key and upload it to the server you want to transfer from.

ssh-keygen -t ed25519
ssh-copy-id -i ~/.ssh/id_rsa.pub [your_old_server.domain.com]

Create a new session and run rsync

tmux new -s [your_session_name]

rsync --avhPW --stats [your_old_server.domain.com]:[/source/path/] [/destination/path]

Detach from session

Press CTRL+B and then D. You session will detach and run in background.

Re-attach session

To get back into your session just:

tmux a -t [your_session_name]

That's it.

Use TouchID on MacBook Pro for Terminal sudo prompts

In order to use TouchID on MacBook Pro for Terminal sudo prompts, we need to enable Apple's Touch ID PAM module pam_tid.so (https://opensource.apple.com/source/pam_modules/pam_modules-173.1.1/modules/pam_tid/pam_tid.c.auto.html).

Just edit /etc/pam.d/sudo and add

auth sufficient pam_tid.so

Heads up, when you do a system update this change will be most probably overwritten. In order to make it persistent, you need to create a launchd daemon.

Create a new file called pam-tid.sh in a shared path

vim /Users/Shared/pam-tid.sh
#!/bin/bash

if ! grep 'pam_tid.so' /etc/pam.d/sudo --silent; then
  sed -i -e '1s;^;auth       sufficient     pam_tid.so\n;' /etc/pam.d/sudo
fi

Create a new com.graffino.pam.plist file:

vim /Users/Shared/com.graffino.pam.plist`

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple Computer//DTD PLIST 1.0//EN" "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
    <key>Label</key>
    <string>com.graffino.pam</string>

    <key>KeepAlive</key>
    <false/>

    <key>LaunchOnlyOnce</key>
    <true/>

    <key>RunAtLoad</key>
    <true/>

    <key>ProgramArguments</key>
    <array>
        <string>/usr/local/bin/pam-tid.sh</string>
    </array>
</dict>
</plist>

Start the daemon