Stephen J Marks

"Do One Thing Well" - Doug Mcllroy

Hello! I’m a Computer Science and Math major with a deep passion for Linux, holding expertise as a Linux specialist. My enthusiasm extends beyond just using the system—I am a committed open source enthusiast, actively contributing to and learning from the community. I am currently seeking a position that allows me to leverage my skills in Linux, particularly in roles related to system administration and DevOps. My professional journey includes significant experience in IT at Managed Service Providers, where I’ve honed my technical and problem-solving skills. I’m also proud to hold certifications from CompTIA and RedHat, which underscore my commitment and expertise in the field. I am excited about the opportunities to bring my technical acumen and passion for Linux to a dynamic team, where I can contribute to meaningful projects and continue to grow in my career.
I want this site to not only be a place where I advertise myself, but can also be myself. So here anything relating to my interests will be found. When I’m not immersed in the digital world, I love stepping into nature. Camping and hiking are my go-to activities for recharging and reconnecting with the outdoors. I also love watching and analyzing films, or consuming material on math, science, or cooking.

Passions 🔥

• Linux & Open Source 🐧
• Films 🎥
• Camping & Hiking 🌲
• Mathematics 🧮
• Cooking & Baking 🍳
• Socializing 🗣️

Links

• My Github: steveonlinux
• This site's repo

Resume 📄

• Downloadable pdf Format

Profile 🖋️

Dynamic system administrator and proficient coder, leveraging a broad knowledge base in technology with sharp programming skills. A dedicated, adaptable enthusiast passionate about advancing the field through open-source contributions.

Degrees and Certifications 🎓

Bachelor's of Computer Science

• Institution: Cal State Bakersfield (In Progress)

Associate of Computer Science & Math

• Institution: Bakersfield College
• Year: 2023

Red Hat Linux Certified Administrator

• Certifying Body: Red Hat
• Year: 2023

CompTIA+ Certifications Itemized

• Certifying Body: CompTIA+
• Years: 2020 - 2022
• Details:
  • Linux+
  • Network+
  • Security+

Employment 🛠️

Library STEM Programming Consultant

• Employer: County of Kern
• Years: 2023 - Current
• Role: Spearheaded and executed educational programs, successfully engaging local youth in computer science, elevating their proficiency in STEM subjects.

IT Support Engineer & Administrator Level 2

• Employer: LanPro Systems
• Years: 2020 - 2022
• Role: Administered Linux VoIP servers, resolved complex tickets, managed AD users, performed system backups, provisioned servers & workstations, and implemented scripts & automated tasks.

Peer Tutor

• Institution: Bakersfield College
• Years: 2019 - 2020
• Role: Led peer instruction in key computer science areas; significantly improving classmates' understanding and performance in programming, logic, and binary arithmetic.

Tutor

• Institutions: Bakersfield College; Tutoring Club; Freelance
• Years: 2019 - Current
• Role: Effectively tutored in computer science, math, and physics; elevating student achievements in college and K-12.

Contact 📞

• Email: stephenmarks5309@gmail.com
• Phone: 661-473-3280
• Website: stevemarks.xyz

Strengths 💪

• Linux
• Programming/Scripting
• System/Server Administration
• Automation (Ansible / K8s / etc)
• Virtualization (KVM / VMware)
• Orchestration (Docker / K8s)
• Provisioning (Ansible / Terraform)
• Integration (Jenkins / Git)
• Text Editing (VIM / VS)

Languages 🌐

• Shell: Fluent
• C: Fluent
• Python: Fluent
• Rust: Basic
• Powershell: Intermediate
• SQL: Basic
• Assembly (x86 / ARM / MIPS): Competent
• AWK: Basic
• HTML + Markdown / CSS: Fluent
• Latex: Intermediate

Works

These are links to my "books" or other writings. My goal is to solidify my own knowledge base while sharing whatever I know with others. I'm majorly self taught and promote self-teaching. Everything here is perpetually in progress.

• Learn To Code! (In Python)

• Calculus Uncovered

Steve's Blog

This will be a blog where I will dump anything I want. It will probably be mostly tech related. Specifically Linux and open source stuff. Thoughts, critiques, criticisms, guides, advice, projects, anything. However, there will probably be some life stuff here too. This will be fun. There will be typos.

Enjoy

On Installing Gentoo For Some Reason (2024-08-21)

I decided to install Gentoo on a whim. I've done it a couple of times in the past, even on Raspberry Pis. But that was before the recent change implementing mass binary alternatives to the compilation Gentoo is known for. Binaries have been available for a while already, but I'm interested to check out this change. Gentoo's biggest con for me is the time it takes to install packages if you don't have a Threadipper. Here I will just be going over my thoughts and experiences. This isn't an actual guide, for that use their handbook for the appropriate architecture.

Where

I decided to install Gentoo in a vm. I went with virtualbox over kvm/qemu because a bridged network will already be setup for ssh and it will be easier to get a shared clipboard going.

Installation of Minimal Linux Distros

Minimal distros like Gentoo and Arch have the essential parts of every distribution, just less obfuscated. Their installation generally has the following cadence:

• Partition drives and create filesystems
• mount root partition
• chroot into root partition
• setup up install in chroot environment
  • network, packages, fstab, bootloader, users, etc.
• post install Chroot is a command that essentially changes the root to a new location. So that everything within the chroot is happening relative to that new root. That is why we chroot into the mounted root partition during installs because at that point we are configuring the actual system to be installed. Take not of this command as it will ultimately help troubleshoot installs from a live cd. Often other paths have to be mounted from the host system to the chroot so that it can be a functioning live environment essentially.

Tips for Installing Gentoo

• Remember to utilize chroot. You don't have to complete the install all at once. Per the handbook, past that point you can always mount your root partition into an environment and with the correct mounts and get back into the system. For there you can continue the installation or fix an issue.
• If you are not installing Gentoo in a GUI already, and you have access to another machine on the network, I recommend remoting into the machine Gentoo is being installed on via SSH. This can be done after being chrooted. Copy and pasting from a web browser makes things so convenient. This will require network bridging
  1. Get network up: net-setup [interface]
  2. Change root's password via passwd
  3. Start the ssh daemon with: /etc/init.d/sshd start
• Take advantage of the newly available binaries for better install times
• Use ccache. At the very least. ccache is a program that caches compiled source so it doesn't have to be recompiled later. Makes later compilations quicker, which is needed on Gentoo.
• Use distcc is you have other computers on the network with good CPUs. distcc distributes compilation workloads across multiple computers. Can make your install a breeze.

My Troubles

• Forgot boot flag on /boot which I caught before it mattered
• fstab typo which caused first boot to be to emergency.target

I fixed both by chrooting back into root and fixing the issue.

Gentoo is fun

I like Gentoo for messing around. It seems like a good educational tool the way LFS can be. Also seems great for development environments. Would never use it on an actual install for a server or desktop though...again. In those instances there is way too much upkeep for what is gained.

Major Packaging Systems Disambiguation 📦 (2024-08-21)

Package Formats

Package formats are the way that software data is organized for distribution prior to installation. Every major Linux branch will have a package format it utilizes. The two most major types are .deb and .rpm. They are found on Debian based distros and Red Hat based distros respectively. Package installers and managers exist for every format and there are typically several for each type that "wrap" or encapsulate another, and sometimes its even in a nested sense. They can be described as having levels such as: "low", "medium", and "high".

.deb

A .deb package is the package used for Debian based distributions. The most major ones being Debian itself and Ubuntu. Deb packages consist three ordered files. Two are tar archives that support compression. One for the metadata known as control information and another for the installable data. The third is a file containing the format version number.

• The structure of a .deb package

dpkg

Dpkg is a low level packaging tool. It isn't a package manager because it doesn't actually manage the packages and dependencies. The dpkg command can be used to install and remove .deb files directly.

• dpkg -i filename.deb installs a package
• dpkg -l lists files that match a pattern
• dpkg -S searches for a filename from installed packages
• dpkg -r packagename removes an installed packages The packages dpkg and dpkg-dev contain the suit of utilities and development tools respectively.

apt-get and apt

Apt-get is a medium level package manager that utilizes dpkg. It can be thought of as a front-end for dpkg. Apt is a wrapper for apt-get that supports greater functionality such as: better dependency resolution, native search capabilities, and autoremoval of orphaned packages during upgrades. Apt is an amalgamation of the functionalities of separate utilities like apt-get and apt-cache. It is recommended that apt-get be reserved for automation. But for the most part apt and apt-get have the same sub-commands and functionality. Apt and Apt-get share the following:

• apt update syncs the package indexes and sources
• apt upgrade installs the current versions of packages
• apt install installs individual or listed packages
• apt clean clears cache of package files
• apt autoclean only clears package files that can no longer be downloaded.

apt exclusive

apt search, apt show, apt policy, apt purge apt also has a set of sub-commands not found in apt-get such as:

• apt search searches the list of packages for a regex term.
• apt show outputs information on a given package

apt Frontends

The are graphical front ends for apt as well like the following:

• synaptic
• aptitude
• Cydia

.rpm

The counterpart to .deb in Red Hat based and upstream distros is the .rpm package format. The file has the naming convention: name-version-release-arch.rpm. This name is appended with -devel for libraries containing two separate packets for precompiled code and the headers. The package is usually provided in binary format and contains:

• The lead, which identifies the file as an RPM file and contains some obsolete headers.
• The signature, which can be used to ensure integrity and/or authenticity.
• The header, which contains metadata including package name, version, architecture, file list, etc.
• A file archive (the payload), which usually is in cpio format, compressed with gzip. The rpm2cpio tool enables retrieval of the cpio file without needing to install the RPM package.

rpm

rpm is the low level counterpart of dpkg for .rpm packages and its purpose and syntax is similar:

• rpm -i packagename.rpm installs a package
• rpm -U packagename.rpm upgrades a package
• rpm -ivh package-file installs a package with verbose output and a progress bar
• rpm -q query-options package searches for an installed package

YUM (Yellow Dog Updater, Modified)

Yellow Dog Updater or YUM is a medium level package manager for Red Hat based distros that acts as an rpm front-end. However, it has since been deprecated.

DNF (Dandified YUM)

Dandified YUM or DNF is the high level package manager that has replaced YUM has the current and up to date rpm front-end. The syntax is similar to apt.

DNF Command Reference

DNF (Dandified YUM) is the package manager for Fedora, CentOS, and RHEL distributions. Here is a list of commonly used commands:

• Update Package Cache:

  dnf check-update
  

• Upgrade All Packages:

  dnf upgrade
  

• Upgrade Specific Package:

  dnf upgrade <package_name>
  

• Install a Package:

  dnf install <package_name>
  

• Remove a Package:

  dnf remove <package_name>
  

• List Available Groups:

  dnf group list
  

• Install a Package Group:

  dnf group install <group_name>
  

• Remove a Package Group:

  dnf group remove <group_name>
  

• Search for a Package:

  dnf search <keyword>
  

• Display Information about a Package:

  dnf info <package_name>
  

• Clean Cache:

  dnf clean all
  

• Auto-remove Unneeded Packages:

  dnf autoremove
  

• List Enabled Repositories:

  dnf repolist
  

• Add a Repository:

  dnf config-manager --add-repo <repository_url>
  

• Enable a Repository:

  dnf config-manager --set-enabled <repository_name>
  

• Disable a Repository:

  dnf config-manager --set-disabled <repository_name>
  
  

Advanced Tips and Tricks

Advanced Tips for apt

1. Full Upgrade vs Safe Upgrade:

   • apt full-upgrade performs upgrades that may also remove installed packages if that's necessary to upgrade the system as a whole.
   • apt upgrade will only upgrade packages without removing any packages.

2. Holding Packages:

   • To prevent a package from being automatically updated, use sudo apt-mark hold package_name.
   • To allow it again, use sudo apt-mark unhold package_name.

3. Searching Package Listings:

   • Use apt-cache search keyword to search for packages. It searches descriptions and names.
   • apt-cache pkgnames starts listing packages alphabetically.

4. List All Versions of a Package Available for Installation:

   • Use apt list -a package_name to see all versions of a package available in your repositories.

5. Using Snapshots:

   • If you want to return your packages to a previous state, you can use apt with snapshots from a backup tool like Timeshift.

6. Automatic Removal of Unused Packages:

   • After uninstalling an application, use sudo apt autoremove to remove dependencies that were installed with applications and that are no longer used by anything else on the system.

7. Find Which Package a File Belongs to:

   • Use apt-file search path/to/file to find out which package a particular file belongs to.

Advanced Tips for dnf

1. History and Undo/Redo Operations:

   • dnf history lists transaction history. You can undo or redo transactions using dnf undo transaction_id or dnf redo transaction_id.

2. Repository Management:

   • Enable or disable repositories on the fly with dnf config-manager --set-enabled repo_name or dnf config-manager --set-disabled repo_name.

3. DNF Automatic:

   • Install dnf-automatic for automatic daily package updates. It can be configured to only download, download and install, or even send email notifications.

4. List and Remove Orphaned Packages:

   • Use dnf list extras to list orphaned packages (packages not available from currently configured repositories).
   • Remove them with dnf remove $(dnf repoquery --extras --exclude='kernel*').

5. Using DNF Plugins:

   • Enhance dnf's capabilities by installing plugins like dnf-plugin-system-upgrade for upgrading Fedora releases, or dnf-plugins-core for additional features like managing core repositories.

6. Cache Management:

   • Use dnf clean dbcache to clean the cache to reduce disk space usage. dnf makecache to regenerate the cache improving the performance of future installations.

7. Find if a Specific Package is Available:

   • Use dnf provides '*/filename' to find which packages provide a specific file.

On Inodes and the Inode Pointer Structure (2024-07-30)

Inodes are a fundamental part of the data structure that stores and references all data available on a storage device.

File Names Are References

The way files and directories are abstracted the end user typically thinks of files being nested. However, this isn't truly the case. Directories are simply lists of inode and file name pairs of the file names underneath, the name of the parent, and the name of itself. An inode is an entity that has a unique number and contains the file metadata, and pointers to the actual data the file "contains". That's it. Inodes do not store their "names" and all the filesystem really cares about are the inodes. File names are essentially just a human readable label mapped to an inode.

What Does An Inode Contain And Reference?

Inodes are where the file's metadata is stored. However, the data itself is not there. The data is referenced through a system of pointers also stored in the inode.

Typical Inode Pointer Structure

The typical pointer structure is implemented in ext* filesystems among others. It has the following qualities:

• *inode pointer structure:
  • 12 direct pointers that directly point to blocks of the file's data
  • 1 singly indirect pointer (pointing to a block of direct pointers)
  • 1 doubly indirect pointer (pointing to a block of single indirect pointers)
  • 1 triply indirect pointer (pointing to a block of doubly indirect pointers)
  • fixed block size (increase redirection when required, data can be stored non contiguously
  • inodes assigned fixed amount at filesystem instantiation, amount of indirect blocks are not fixed
  • number of pointers dependent on block size and size of pointers The pointers are set up to use direct pointers and others with varying levels of indirection. This is so that the structure can support varying amounts of data while remaining static. If more space is needed, the system can simply use any unclaimed data block at some far off address through redirection.

Pointers and Metadata

Here is a diagram of the structure of inodes:

You can see the structure of the pointers that were previously discussed. However, the undiscussed metadata section is present as well. This section contains metadata such as filer permissions, ownership, edit history, etc. An exact list of metadata in an inode will be provided in the next subsection.

Viewing Contents of Inodes

The contents of inodes can be viewed using the stat system call.

stat test.txt
  File: test.txt
  Size: 15        	Blocks: 8          IO Block: 4096   regular file
Device: 259,2	Inode: 30158006    Links: 3
Access: (0644/-rw-r--r--)  Uid: ( 1001/   steve)   Gid: ( 1001/   steve)
Access: 2024-07-26 08:23:43.390310050 -0700
Modify: 2024-07-26 08:24:02.074309630 -0700
Change: 2024-07-28 13:29:14.971830641 -0700
 Birth: 2024-07-26 08:23:43.390310050 -0700

This command lists the contents of the inode test.txt is paired to. The metadata an inode contains includes:

• File Type: Indicates the type of file (e.g., regular file, directory, symbolic link).
• Permissions: Specifies read, write, and execute permissions for the owner, group, and others.
• Owner User ID (UID): The user ID of the file's owner.
• Owner Group ID (GID): The group ID of the file's owner group.
• File Size: The size of the file in bytes.
• Number of Hard Links: The count of hard links pointing to this inode.
• Timestamps:
  • Access Time (atime): Last time the file was accessed.
  • Modification Time (mtime): Last time the file content was modified.
  • Change Time (ctime): Last time the inode metadata was changed.
• File Size in Blocks: The number of blocks allocated to the file.
• Extended Attributes: Additional metadata such as security labels or custom attributes.
• Flags: File flags indicating special characteristics or behaviors (e.g., immutable, append-only).
• Generation Number: A number that changes each time the inode is reused, used to detect stale file handles.
• File System Identifier: An identifier for the file system where the inode resides.
• The device ID of the file if it is a device file. Links are discussed in the next section.

Hard Links and Soft Links

Hard Links Are Names

File names are known as hard links. And the nature of the inode structure inherently allows files to have multiple hard links. the -i option on various commands outputs information on a file's inodes.

30158006 .rw-r--r-- 15 steve 26 Jul 08:24 test.txt

Here ls -li long lists with the inode number. So test.txt is mapped to inode 30158006. To create a hard link the command ln is used.

ln test.txt hard_link_to_test.txt

This command creates a hard link named hard_link_to_test.txt to test.txt.

ls -li test.txt hard_link_to_test.txt 
30158006 .rw-r--r-- 15 steve 26 Jul 08:24 hard_link_to_test.txt
30158006 .rw-r--r-- 15 steve 26 Jul 08:24 test.txt

Here we can see that these two files share the same inode number: 30158006. This means that both file names are mapped to that inode and thus are referencing the same data.

Soft Links Are Pointers To Names

There is also the concept of a soft link aka a symbolic link. These links are file names that are pointers to other names. A soft link can be created with the -s option as in ln -s.

ln -s test.txt soft_link_to_test.txt

This command creates a soft link named soft_link_to_test.txt to test.txt.

ls -li test.txt hard_link_to_test.txt soft_link_to_test.txt
30158006 .rw-r--r-- 15 steve 26 Jul 08:24 hard_link_to_test.txt
30157667 lrwxrwxrwx  8 steve 28 Jul 13:37 soft_link_to_test.txt -> test.txt
30158006 .rw-r--r-- 15 steve 26 Jul 08:24 test.txt

Here we can see that the soft link does not share the inodes of the hard links but has its own. That is because it is pointing to the file name test.txt and not mapped to that filename's inode as a hard link would be. A soft link is mapped to its own inode, and that inode contains the path of the file being pointed to. To unlink a link the unlink command can be used.

 unlink hard_link_to_test.txt 
╭─steve@kathy-office ~/test
╰─❯ unlink soft_link_to_test.txt 
╭─steve@kathy-office ~/test
╰─❯ ls -li test.txt hard_link_to_test.txt soft_link_to_test.txt
"hard_link_to_test.txt": No such file or directory (os error 2)
"soft_link_to_test.txt": No such file or directory (os error 2)
30158006 .rw-r--r-- 15 steve 26 Jul 08:24 test.txt

Link Use Cases

Hard Links

I've seen hard links described as a "poor man's backup" which means that if one filename is deleted the inode isn't inaccessible. There is still some other name mapped to that inode. When a file is deleted via something like rm, the data still exists. This will be covered in a later section. This also saves space, if what is hard linked in large, but needs to exists in multiple paths. And since inodes store metadata such as permissions and file attributes, the hard link will share them as well.

Soft Links

Soft links allow for having files accessible from multiple paths without moving or copying anything. Another use is version management. Say there is one service or script that runs and depends on something having a certain path or name. but it changes regularly. A soft link to it won't. So instead of rewriting anything, the soft link can just be redirected. Soft links also have the capability of linking to files across filesystems and devices, where hard links can't.

Restrictions

Linking has some limitations due to the inherent structure of filesystems and inodes.

Linking To Directories

Directories can not be hard linked in Linux. Historically this has been possible, but most operating systems disable this behavior now to avoid cycles. A notable counterexample is that a superuser can make hard links to directories in MacOS 10.5. Directories can be soft linked because a soft link's inode just contains the path of the file being pointed to.

Inode Exhaustion

Filesystems have a limited and fixed number of inodes for use, and all data stored needs an inode pointing to it. If one created countless small files containing nothing each would require an inode and they would eventually run out, and no more files can be created. This is known as inode exhaustion. And since the files were empty there wold still be ample free space, and everything would appear fine on the surface. The number of available inodes can be checked using df -i.

df -i
Filesystem     Inodes IUsed IFree IUse% Mounted on
udev             1.7M   535  1.7M    1% /dev
tmpfs            1.7M   936  1.7M    1% /run
/dev/nvme0n1p2    58M  606K   57M    2% /
tmpfs            1.7M   712  1.7M    1% /dev/shm
tmpfs            1.7M     6  1.7M    1% /run/lock
tmpfs            1.7M    94  1.7M    1% /tmp
/dev/nvme0n1p1      0     0     0     - /boot/efi
tmpfs            348K   106  348K    1% /run/user/1001

Based on this output we can see I have plenty of inodes free.

Setting Number of Inodes in Filesystem

Since the amount of inodes are fixed the only way to alter it is to set it at filesystem creation. Say we are creating an ext4 filesystem. The mkfs.ext4 can do this. The following is from its man page: >-N number-of-inodes Overrides the default calculation of the number of inodes that should be reserved for the file system (which is based on the number of blocks and the bytes-per-inode ratio). This allows the user to specify the number of desired inodes directly.

Note that there are some exceptions. For instance Btrfs is capable of dynamically allocating inodes so the amount is not fixed.

Alternative Structures and Filesystems

Many filesystems exist that are fundamentally different that something like ext*. They do not use inodes but utilize other data structures such as B-trees. The following is a list of filesystems that do not use inodes:

• FAT (File Allocation Table): Used by MS-DOS and early versions of Windows, FAT file systems, including FAT12, FAT16, and FAT32, use a file allocation table to manage files rather than inodes.
• exFAT (Extended File Allocation Table): Designed by Microsoft to replace FAT32, exFAT is optimized for flash drives and does not use inodes.
• ReFS (Resilient File System): Developed by Microsoft, ReFS uses B-trees for metadata storage and does not use inodes.
• HFS (Hierarchical File System): Used in older Apple Macintosh systems, HFS uses a catalog file and extents overflow file for metadata instead of inodes.
• HFS+ (Hierarchical File System Plus): An updated version of HFS used in macOS before APFS, HFS+ uses B-trees for metadata storage, avoiding the inode structure.
• APFS (Apple File System): The current file system for macOS and iOS, APFS uses container and volume structures with object identifiers instead of inodes.
• NTFS (New Technology File System): Used by modern versions of Windows, NTFS uses a Master File Table (MFT) to store metadata rather than inodes.
• IBM GPFS (General Parallel File System): Now known as Spectrum Scale, GPFS uses a distributed metadata model with descriptors rather than traditional inodes.

Retrieving Unclaimed Data

Recall that when a hard link is deleted via rm the data still exists, however once the last hard link to the inode is gone the metadata is deleted and there is no way to access the data. The data still exists but the space it occupies is now unclaimed can be reclaimed and overwritten by new data. If data needs to be recovered cease all writes immediately. Either unmount the drive or boot into a live environment. The drive can not be written to until the data is located. There are a number of programs that can inspect data fields for information indicating the types of data stored there. They may utilize a technique of data recovery known as file carving in which data is reconstituted from fragments.

In conclusion, understanding inodes is a critical part of understanding linking and the filesystems used on Linux.

Resources

Inode Pointer Structure

Inodes

Inodes

Linking

Unix File System

On Pseudo Devices (2024-07-30)

One of the most interesting and powerful design philosophies behind Unix and its kin is the notion of everything being a file. Everything. For instance the kernel state is is abstracted as files in /proc, and access to data in physical devices is abstracted as files in /dev. Files is /dev are known as special files. There is a kind of special file called pseudo devices. These kinds of files don't necessarily abstract access to physical devices, but abstract data and its manipulation as files. I find this concept incredibly fascinating and is one of the reasons I love using and learning Linux. This post will be on pseudo devices - what they are, why they exist, and how they are and can be used.

Available Pseudo devices in Linux

Pseudo devices can be found in /dev. Here is a list of all the pseudo devices available in Linux:

• /dev/null - A special file that discards all data written to it and provides zero bytes when read.
• /dev/zero - Provides a continuous stream of null (zero value) bytes.
• /dev/random - Provides random data generated from environmental noise.
• /dev/urandom - Similar to /dev/random but uses a pseudo-random number generator to provide data.
• /dev/full - Always returns an error indicating that the device is full when writing to it.
• /dev/ptmx - Used to create a new pseudo-terminal master and slave pair.
• /dev/tty - Represents the controlling terminal for the current process.
• /dev/console - Represents the system console.
• /dev/loopX - Loopback devices that allow a file to be mounted as a filesystem.
• /dev/ppp - Represents a point-to-point protocol network device.
• /dev/ttyS* or /dev/ttyUSB* or /dev/ttyACM* - Serial devices.
• /dev/net/tun and /dev/net/tap - Network tunnel devices for tunneling IP traffic (TUN) and Ethernet frames (TAP).
• /dev/fuse - Filesystem in Userspace, allows non-privileged users to create their own filesystem without modifying kernel code.
• /dev/shm - Represents a temporary filesystem for shared memory.
• /dev/mem - Provides access to the physical memory of the system.
• /dev/kmsg - Used for logging kernel messages.
• /dev/bus/usb - Represents USB devices and buses.
• /dev/disk/by-id - Symbolic links to storage devices based on their IDs.
• /dev/disk/by-path - Symbolic links to storage devices based on their physical paths.
• /dev/input/event* and /dev/input/mice - Represents input devices like keyboards, mice, and other human interface devices.

Most of these devices serve the system, however there are pseudo devices available that can provide certain functionalities in the cli and scripts.

Data Manipulation

/dev/null/ can be used as a trash can basically. Any data you don't need can be streamed to this device as if you're chucking it into a black hole never to be see again. It's most often sued as a place to redirect either stdout or stderr in scripts or one-liners.

# Run the command in the background and discard its output
command > /dev/null 2>&1 &

dev/zero...outputs a stream of zeroes. This is often used for wiping out a file or drive. Here is a line that does this by writing to a file, and once the drive is full, the file is deleted.

        dd if=/dev/zero of=zeros; rm zeros

This can be part of a larger script used for backing up drives.

#Fill up the blank space on drive with zeros first (optional):

        dd if=/dev/zero of=zeros; rm zeros

#Recommended: Unmount drive before backing it up.

#Creating the archive:

        dd if=/dev/sda1 conv=noerror | gzip -c > sda1.img.gz

#Extracting the archive:

        cat sda1.img.gz | gzip -dc | dd of=/dev/sda1 conv=noerror

#Storing disk info:

        fdisk /dev/sda -l > sda1.txt

Random Data Generation

/dev/random or dev/urandom generates a random stream of data. This data can be dumped to overwrite drives with random data, or generate very strong passwords. Note that the c functions random(), srandom(), and the system call getrandom() are also at your disposal.

#!/bin/bash

# Generate the password
PASSWORD=$(head /dev/urandom | tr -dc A-Za-z0-9 | head -c "$1")

echo "Generated password: $PASSWORD"

This small script generates a random password of a length given as a command line argument.

#!/bin/bash

# Check if the user provided a drive as an argument
if [ $# -eq 0 ]; then
    echo "Usage: $0 /dev/sdX"
    echo "Example: $0 /dev/sda"
    exit 1
fi

# Drive to overwrite
DRIVE=$1

# Confirm with the user before proceeding
read -p "WARNING: This will irreversibly overwrite all data on $DRIVE. Are you sure? (yes/no): " CONFIRM
if [ "$CONFIRM" != "yes" ]; then
    echo "Operation cancelled."
    exit 1
fi

# Overwrite the drive with random data
echo "Overwriting $DRIVE with random data. This may take a while..."
dd if=/dev/urandom of=$DRIVE bs=1M status=progress

echo "Overwrite complete."

This one overwrites a drive with random data.

Always take advantage of pseudo devices, especially /dev/null in scripts and the cli. There are other pseudo devices we use on Linux such as pseudoterminals, serial devices, and even pipes themselves are considered pseudo devices. I recommend reading up on these as well.

Links

Device Files

/dev/random

Bootloaderless Boot With EFIstub (2024-07-25)

The most prevalent bootloader on Linux is GRUB2, but there are other offerings put there such as LIFO. But did you know that...you don't need a bootloader at all?

UEFI

UEFI, the replacement of the legacy BIOS, is actually capable of selecting and loading kernels directly, which I don't see many people doing. You don't actually need a booatloader with UEFI. The downside is that if you would like to boot other OSs, you'll have to access the UEFI every time you would like to boot something else. But if you only have a single OS installed, I don't see any downsides.

EFIstub

This can be achieved with an EFIstub. An EFIstub is essentially a kernel that is executable by EFI. This may already be built into the kernel you're using if CONFIG_EFI_STUB=y is set. If not... well compiling the kernel will be its own post. The UEFI essentially just needs an entry for the kernel that is being booted. You could do this form withing your UEFI's shell, but I use efibootmgr for this. It can manage UEFI boo entries from within Linux. If the UEFI is configured appropriately and its version supports it, booting the kernel shouldn't be an issue. Note that keeping the kernel that is being booted updated automatically is distro specific so refer to your distro's documentation. Arch's states:

pacman will directly update the kernel that the EFI firmware will read if you mount the ESP to /boot.

So that's what I did.

efibootmgr

Setting the UEFI boot entry is jsut a single command. I wrote a small script that can be used to automate the process. The script sets it up using the UUID of the root partition. It creates an entry and deletes the previous one if it already exists.

#!/bin/sh

# Delete the existing boot entry
efibootmgr -b 0 -B > /dev/null
printf "Deleted boot entry: 0000\n\n"

# Retrieve the UUID for the root partition
ROOT_UUID=$(sudo blkid | grep root | awk -F 'UUID="' '{split($2, a, "\""); print a[1]}')

# Reload the entry with the new UUID
printf "Reloaded entry 0000...\n"
efibootmgr --disk /dev/nvme0n1 --part 1 --create --label "Arch Linux" --loader /vmlinuz-linux --unicode "root=UUID=$ROOT_UUID rw initrd=\initramfs-linux.img rd.driver.blacklist=nouveau nvidia-drm.modeset=1" --verbose --timeout 0 | grep "Boot0000"

After this I select my kernel in the UEFI boot config and I'm good to go. If you use this script remember to use your own kernel boot parameters if you need them and to use the path to the correct disk. In my case its /dev/nvme0n1.

You should have a fast boot and a good time.

Built-ins, Utilities, Functions, and System Calls Disambiguation (2024-07-25)

The differences between these haven't always been clear to me in the context of Linux. Many shared names also caused me some confusion. I'm assuming this is the case for others and an explanation may serve them.

Core Utilities

Core utilities or coreutils are just sets of programs commonly installed on systems by default. The most common and ubiquitous set of coreutils are are the GNU coreutils. If you're on Linux, you're probably using them. The coreutils are discrete binaries that can each be ran for basic and essential functionality required to use a system. For example, ls and mv are coreutils. There are alternative sets of coreutils such as uutils, which are the gnu ones rewritten in Rust. There is also Busybox which are coreutils reimplemented as a single binary. Busybox is usually found on distributions of Linux like Alpine that are extremely small intended for embedded environments. And other operating systems have their equivalents such as the BSDs. In fact there are some that I personally prefer over gnu counterparts such as doas.

Shell Built-ins

The first bit of confusion arises with the introduction of shell built-ins. A shell being an environment used to interact with the components and indirectly the kernel of the operating system; a built-in is a command that is in the shell itself. It is not its own program or binary and does not exist anywhere on the system. The which command can tell you if a command is built-in or not. which outputs the path of executable files, so if you run a command as an argument and a path isn't output, its built-in.
The printf command is a case where there is likely both a binary installed and a shell built-in (at least in the case of bash). To force the use of the binary one would just run the path to it. For example:

 /usr/bin/printf "Hello World!\n"
Hello World!

To force the use of the built-in the command builtin (which funnily enough is itself built-in) can be ran with the desired built-in command as an argument. For example:

builtin printf "Hello World!\n"
Hello World!

By default the binary is used.

C Functions

You may have noticed that printf shares the name of the c standard function c printf(). At the end of the day almost everything in Linux is ultimately C and the printf command is a kind of wrapper that just invokes the c function to output and format text. glibc is the gnu implementation of the c standard library which is, like the coreutils, on the vast majority of Linux distributions.

System Calls

Like commands that share the names of c functions, they can also share the names of system calls. System calls a c function that utilizes the kernel API to interface with the kernel. There exists c wrapper functions for system calls invoke the system call. Let's use the wait command as an example going down the layers of abstraction from top to bottom.

wait Built-in

The wait command is a built-in that detects changes in the status of a process.

$ sleep 10 &
$ jobs -l
[1] + 18057 Running              sleep 10
$ wait 18057
[1] + Done                       sleep 10
$

wait terminates once the job ends.

wait C Function

The wait() function in the c standard library is what the wait built-in is effectively wrapping.

#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>

int main() {
    pid_t pid = fork();

    if (pid == -1) {
        perror("fork");
        exit(EXIT_FAILURE);
    } else if (pid == 0) {
        // Child process
        printf("Child process is running\n");
        sleep(1);  // Simulate some work with sleep
        exit(EXIT_SUCCESS);
    } else {
        // Parent process
        wait(NULL);  // Wait for the child to terminate
        printf("Child process has terminated\n");
    }

    return 0;
}

The above program contains fork() and wait(), two c functions that wrap system calls. This code demonstrates the use of wait() as a c function.

wait System Call

#include <sys/types.h>
#include <sys/wait.h>
#include <unistd.h>
#include <stdio.h>
#include <stdlib.h>
#include <sys/syscall.h>

int main() {
    pid_t pid = fork();

    if (pid == -1) {
        perror("fork");
        exit(EXIT_FAILURE);
    } else if (pid == 0) {
        // Child process
        printf("Child process is running\n");
        sleep(1);  // Simulate some work with sleep
        exit(0);
    } else {
        // Parent process
        syscall(SYS_wait4, -1, NULL, 0, NULL);  // Wait for the child to terminate
        printf("Child process has terminated\n");
    }

    return 0;
}

The above code utilizes syscall() which is a function that accepts system calls themselves as parameters. SYS_wait is the actual system call. And all system calls share the same naming scheme.

System Calls All The Way Down

So every command - whether it be a binary or built-in - issued is ultimately some c code and a system call. And many share the same name! So when you're reading some documentation or a man page, understand the distinctions between them, or how there all parts of a greater whole at times abstracting one another. Then there is no need to be confused.

Some System Calls

• SYS_read: Read from a file descriptor.
• SYS_write: Write to a file descriptor.
• SYS_open: Open a file.
• SYS_close: Close a file descriptor.
• SYS_stat: Get file status.
• SYS_fstat: Get file status of an open file.
• SYS_lstat: Get file status of a symbolic link.
• SYS_poll: Wait for some event on a file descriptor.
• SYS_lseek: Reposition read/write file offset.
• SYS_mmap: Map files or devices into memory.
• SYS_mprotect: Set protection on a region of memory.
• SYS_munmap: Unmap a region of memory.
• SYS_brk: Change data segment size.
• SYS_rt_sigaction: Examine and change a signal action.
• SYS_rt_sigprocmask: Examine and change blocked signals.
• SYS_ioctl: Control device.
• SYS_pread64: Read from a file descriptor at a given offset.
• SYS_pwrite64: Write to a file descriptor at a given offset.
• SYS_readv: Read data into multiple buffers.
• SYS_writev: Write data from multiple buffers.
• SYS_access: Check user's permissions for a file.
• SYS_pipe: Create a pipe.
• SYS_select: Synchronously monitor multiple file descriptors.
• SYS_sched_yield: Yield the processor.
• SYS_mremap: Remap a virtual memory address.
• SYS_msync: Synchronize a file with a memory map.
• SYS_mincore: Determine whether pages are resident in memory.
• SYS_madvise: Give advice about use of memory.
• SYS_shmget: Get a System V shared memory segment.
• SYS_shmat: Attach a System V shared memory segment.
• SYS_shmctl: Control operations on a shared memory segment.
• SYS_dup: Duplicate a file descriptor.
• SYS_dup2: Duplicate a file descriptor to a given descriptor.
• SYS_pause: Wait for a signal.
• SYS_nanosleep: High-resolution sleep.
• SYS_getitimer: Get the value of an interval timer.
• SYS_alarm: Set an alarm clock for delivery of a signal.
• SYS_setitimer: Set the value of an interval timer.
• SYS_getpid: Get process ID.
• SYS_sendfile: Transfer data between file descriptors.
• SYS_socket: Create an endpoint for communication.
• SYS_connect: Initiate a connection on a socket.
• SYS_accept: Accept a connection on a socket.
• SYS_sendto: Send a message on a socket.
• SYS_recvfrom: Receive a message from a socket.
• SYS_sendmsg: Send a message on a socket.
• SYS_recvmsg: Receive a message from a socket.
• SYS_shutdown: Shut down part of a full-duplex connection.
• SYS_bind: Bind a name to a socket.
• SYS_listen: Listen for connections on a socket.
• SYS_getsockname: Get socket name.
• SYS_getpeername: Get name of connected peer socket.
• SYS_socketpair: Create a pair of connected sockets.
• SYS_setsockopt: Set options on sockets.
• SYS_getsockopt: Get options on sockets.
• SYS_clone: Create a child process.
• SYS_fork: Create a child process.
• SYS_vfork: Create a child process and block parent.
• SYS_execve: Execute program.
• SYS_exit: Terminate the calling process.
• SYS_wait4: Wait for process to change state.
• SYS_kill: Send a signal to a process.
• SYS_uname: Get system information.
• SYS_semget: Get a System V semaphore set.
• SYS_semop: Operate on a System V semaphore set.
• SYS_semctl: Control a System V semaphore set.
• SYS_shmdt: Detach a System V shared memory segment.
• SYS_msgget: Get a System V message queue.
• SYS_msgsnd: Send a message to a System V message queue.
• SYS_msgrcv: Receive a message from a System V message queue.
• SYS_msgctl: Control operations on a System V message queue.
• SYS_fcntl: Manipulate file descriptor.
• SYS_flock: Apply or remove an advisory lock on an open file.
• SYS_fsync: Synchronize a file's in-core state with storage device.
• SYS_fdatasync: Synchronize a file's in-core state with storage device.
• SYS_truncate: Truncate a file to a specified length.
• SYS_ftruncate: Truncate an open file to a specified length.
• SYS_getdents: Get directory entries.
• SYS_getcwd: Get current working directory.
• SYS_chdir: Change working directory.
• SYS_fchdir: Change working directory of the calling process.
• SYS_rename: Change the name or location of a file.
• SYS_mkdir: Create a directory.
• SYS_rmdir: Remove a directory.
• SYS_creat: Create a file.
• SYS_link: Create a new name for a file.
• SYS_unlink: Delete a name and possibly the file it refers to.
• SYS_symlink: Create a symbolic link.
• SYS_readlink: Read value of a symbolic link.
• SYS_chmod: Change permissions of a file.
• SYS_fchmod: Change permissions of an open file.
• SYS_chown: Change ownership of a file.
• SYS_fchown: Change ownership of an open file.
• SYS_lchown: Change ownership of a file, not following symbolic links.
• SYS_umask: Set file mode creation mask.
• SYS_gettimeofday: Get time.
• SYS_getrlimit: Get resource limits.
• SYS_getrusage: Get resource usage.
• SYS_sysinfo: Get system information.
• SYS_times: Get process times.
• SYS_ptrace: Trace a process.
• SYS_syslog: Read and/or clear kernel message ring buffer; set console_loglevel.
• SYS_setuid: Set user identity.
• SYS_setgid: Set group identity.
• SYS_setpgid: Set process group ID.
• SYS_getppid: Get parent process ID.
• SYS_getpgrp: Get process group ID.
• SYS_setsid: Create a session and set process group ID.
• SYS_setreuid: Set real and effective user IDs.
• SYS_setregid: Set real and effective group IDs.
• SYS_getgroups: Get list of supplementary group IDs.
• SYS_setgroups: Set list of supplementary group IDs.
• SYS_setresuid: Set real, effective, and saved user IDs.
• SYS_getresuid: Get real, effective, and saved user IDs.
• SYS_setresgid: Set real, effective, and saved group IDs.
• SYS_getresgid: Get real, effective, and saved group IDs.
• SYS_getpgid: Get process group ID.
• SYS_setfsuid: Set user ID used for filesystem checks.
• SYS_setfsgid: Set group ID used for filesystem checks.

gnu++ in Rust (2024-07-24)

I've discussed rewrites of commonly used gnu utilites in Rust. I've written a script that installs cargo and every Rust rewrite with it. The script also optionally writes alias definitions to the appropriate shell rc file. It assumes bash by default. The script source will be pasted below, and the repo is on my Github.

Source

#!/bin/bash

# Default list of crates
default_crates=(
    "bat"
    "choose"
    "exa"
    "lsd"
    "git-delta"
    "du-dust"
    "duf"
    "broot"
    "fd-find"
    "ripgrep"
    "sd"
    "bottom"
    "gping"
    "procs"
    "zoxide"
    "dns-doge"
    "mcfly"
    "xh"
    "hyperfine"
)

# Default shell config file
shell_config=".bashrc"

# Usage function
usage() {
    echo "Usage: $0 [-a] [-s SHELL_CONFIG] [-f PACKAGE_FILE] [-h]"
    echo "  -s: Specify shell configuration file (.bashrc or .zshrc)"
    echo "  -f: Specify a file with a list of packages"
    echo "  -h: Display this help message"
    exit 1
}

flag=0
# Parse command-line options
while getopts ":as:f:h" opt; do
    case $opt in
        a)
            flag=1
            ;;
        s)
            shell_config="$OPTARG"
            ;;
        f)
            package_file="$OPTARG"
            ;;
        h)
            usage
            ;;
        \?)
            echo "Invalid option: -$OPTARG" >&2
            usage
            ;;
    esac
done

# Read packages from file if specified
if [ -n "$package_file" ]; then
    if [ -f "$package_file" ]; then
        mapfile -t default_crates < "$package_file"
    else
        echo "Error: Package file not found: $package_file"
        exit 1
    fi
fi

install_rust() {
  curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
  PATH="$HOME/.cargo/bin:$PATH"
  printf "\n" >> "$HOME/$shell_config"
  echo "export PATH=$PATH" >> "$HOME/$shell_config"
}

# Function to set up aliases
setup_aliases() {
    {
        printf "\n"
        echo "#gnu++ Rust Alternative Aliases"
        echo "alias cat='bat --paging=always'"
        echo "alias ls='exa -la'"
        echo "alias diff='git-delta'"
        echo "alias du='du-dust'"
        echo "alias df='duf'"
        echo "alias tree='broot'"
        echo "alias find='fd'"
        echo "alias grep='rg'"
        echo "alias find='fd'"
        echo "alias cut='choose'"
        echo "alias sed='sd'"
        echo "alias top='btm'"
        echo "alias ps='procs'"
        echo "alias cd='z'"
        echo 'eval "$(zoxide init bash)"'
        echo "alias dig='doge'"
    } >> "$HOME/$shell_config"
    echo "Aliases added to $HOME/$shell_config"
}

# Install packages and set up aliases
install_rust_crates() {
    for package in "${default_crates[@]}"; do
        cargo install --locked "$package"
    done
}

main() {
install_rust
install_rust_crates
if [ "$flag" -eq 1 ]; then
  setup_aliases
fi
echo "Installation complete."
}

main

Usage

Usage: ./gnu++.bash [-a] [-s SHELL_CONFIG] [-f PACKAGE_FILE] [-h]
  -s: Specify shell configuration file (.bashrc or .zshrc)
  -f: Specify a file with a list of packages
  -h: Display this help message

What I Use 🐧 (2024-07-23)

I realized I've yet to discuss what it is I use day to day in terms of Linux. This is a quick post dedicated to my setup.

Desktops/Laptops

On the desktop and I use Arch Linux. Arch is a rolling release distribution which means packages, including the kernel, are the latest available versions... which can lead to some issues. I've had small issues over the years using it mostly relating to dependencies breaking on updates. Usually manually holding back a package in /etc/pacman.conf fixes these issues. Random issues like some files not being deleted properly by the package manager preventing an update also arise but usually the issue can be manually fixed by reading an error message and investigating. All that being said the nature of Arch means that package availability is incredible. Not to mention the aur or Arch User Repository which is a repo consisting of packages packaged by users and developers themselves. While these may be inherently less trustworthy than official packages, I've almost never run into a situation where what I as looking for wasn't available on Arch.

Window Manager

My previously favorite window manager was dwm. I liked dwm but the philosophy involving manually patching source is very tedious and arbitrary. It seemed like core functionality was missing by default. So I decided to switch to hyprland because I wanted to switch to Wayland as my display server...protocol. Hyprland is technically a compositor that manages windows. Wayland has been amazing for me. Beforehand I was plagued by screen tearing on Xorg. However, I had to get an AMD gpu for my desktop for Wayland compatibility, but I needed to upgrade that anyway. Hyprland is amazing too. It is fast, clean, and offers amazing functionality. Very cool animations, great defaults, and the config file is easy to read and edit.

Servers

For server I either use Debian, Rocky Linux, or PiOS if I'm running things off of a Pi.

Dotfiles

My dotfiles are available on my Github.

Raspberry Pi at Home 🥧 (2024-07-22)

Let's Take a Trip To Pie Country

What is an SBC?

A SBC or single board computer is a computer...on a single board. All the components, CPU, RAM, flash, i/o, on a single PCB or printed circuit board. They're usually really small and relatively inexpensive. They're commonly used for small hobbyist projects because they often have buses one can just shove raw wires into, they're small enough to cram into whatever you want, and have relatively low power usages. They are commonly associated with edge computing and IoT or internet of things. These are two interrelated concepts involving smart fridges and toasters. Thermostats, air quality sensors, and home security systems. Washers and dryers. Even soldering irons and your girlfriend's hair iron. Devices containing tiny computers connected to the internet, sometimes storing information and doing computations on a user's local infrastructure.

Buying Options

There are many brands to choose from:

However, the market is dominated by the Raspberry Pi. It is the de facto option and is synonymous with the term SBC. There are a number of models to choose from:

Why Raspberry Pi?

It being the de facto option it has the most support. The Pi project has: a Linux distribution, hardware /software support, devices such as sensors and monitors, and forums. And the massive slice of the market Pi has means third party support is substantial as well. There are countless add on boards or HATs and enclosures that can be had. No matter the fact that competing SBCs may have comparable or even surpassing performance capabilities, many will choose Raspberry Pi due the overwhelming support. The greatest con is the availability. Raspberry Pis have historically been extremely difficult to come by. Especially considering memory shortages, and global pandemic, and a chip shortage over the last few years, the 4th model was essentially impossible to get, especially at retail. There are entire databases dedicated to tracking retailer stocks. However, the release of the 5th model placated most and made obtaining at a reasonable price possible.

What I Use Them For

Again there are a plethora of way to utilize a Pi due to the sheer amount of openness and attachable boards and devices. I personally use them in my homelab. I use them to host a variety of services, usually through containerization such as Docker. My Raspberry Pi 4s and 5s serve:

• DNS through PiHole
• Dynamic DNS through ddclient and DuckDNS
• VPNs through PiVPN
• IP KVM using the PiKVM board
• File sharing through Samba
• Media through Plex
• Password management through Passbolt
• Site hosting through Apache and Nginx

A freshly arrived piKVM

That piKVM running booted off of a USB

I've used containers to run things like this as often as possible. Many utilities are available as container images on Dockerhub, however mileage may vary as ARM image are less common. I prefer containers due to the contained nature of the binaries and configurations. Everything can be contained in a single docker-compose file, which is very convenient. In the past I've even had my Pis clustered using Kubernetes, but that was more for educational purposes. I've also experimented with different Linux distributions on Pis. There is of course Pi OS, the officially supported one. However,the major distributions have ARM versions that run on Pis and there are others specifically designed for Pis:

For instance installing and running Gentoo on one was... an experience.

All in all, Pis are a great way to learn and an even better way to run services at home at a low monetary cost and low power cost.

Over The Wire (Bandit) 🧑‍💻 (2024-07-17)

Link

This is a wargame or ctf game that has the player interact with and penetrate a Linux system. The goal of each level is to retrieve a password or key located somewhere on a server. This can be used to access this next level, which is a user on the server.

SPOILER WARNING

I will provide the prompt and hints to each level and my answers. I may describe my methodology, reasoning, or solution if it is warranted. Do not read if you intend on playing yourself.

Level 0

The goal of this level is for you to log into the game using SSH. The host to which you need to connect is bandit.labs.overthewire.org, on port 2220. The username is bandit0 and the password is bandit0. Once logged in, go to the Level 1 page to find out how to beat Level 1.

Solution

• ssh bandit0@bandit.labs.overthewire.org -p 2220
• Once logged in cat readme will output password to level 1.
• Password: ZjLjTmM6FvvyRnrb2rfNWOZOTa6ip5If

Level 1

The password for the next level is stored in a file called - located in the home directory

• ssh bandit1@bandit.labs.overthewire.org -p 2220
• The name of the file containing the password needs to be referenced as a file as - is reserved for other uses in the shell.
• cat ./- will do the trick
• Password: 263JGJPfgU6LtdEvgfWU1XP5yac29mFx

Level 2

The password for the next level is stored in a file called spaces in this filename located in the home directory

• ssh bandit2@bandit.labs.overthewire.org -p 2220
• This is a similar issue as whitespace has a reserved use is shells as well. The file must be referenced by encasing it with quotes, cat "spaces in this filename" or by escaping the spaces, cat spaces\ in\ this\ filename.
• Password: MNk8KNH3Usiio41PRUEoDFPqfxLPlSmx

Level 3

The password for the next level is stored in a hidden file in the inhere directory.

• ssh bandit3@bandit.labs.overthewire.org -p 2220
• cat inhere/...Hiding-From-You
• Password: 2WmrDFRmJIq3IPxneAaMGhap0pFhF3NJ

Level 4

The password for the next level is stored in the only human-readable file in the inhere directory. Tip: if your terminal is messed up, try the “reset” command.

• ssh bandit4@bandit.labs.overthewire.org -p 2220
• This is the first level that doesn't essentially have a single solution. Essentially do whatever you need to to determine which file is human readable or cat everything to a file and sift through garbage until you see the password.
• Using file and a wildcard:

bandit4@bandit:~$ file inhere/*
inhere/-file00: data
inhere/-file01: data
inhere/-file02: data
inhere/-file03: data
inhere/-file04: data
inhere/-file05: data
inhere/-file06: data
inhere/-file07: ASCII text
inhere/-file08: data
inhere/-file09: data

Looks to be inhere/-file07.

bandit4@bandit:~$ cat inhere/-file07
4oQYVPkxZOOEOO5pTW81FB8j8lxXGUQw

Level 5

The password for the next level is stored in a file somewhere under the inhere directory and has all of the following properties: human-readable 1033 bytes in size not executable

• Most automated solution is probably just using find
• find . -size 1033c \! -executable

bandit5@bandit:~$ find . -size 1033c \! -executable   
./inhere/maybehere07/.file2

• Can cat this in a one liner by running find in a subshell:

bandit5@bandit:~$ cat $(find . -size 1033c \! -executable)
HWasnPhtq9AVKe0dmk45nxy20cvUa6EG

Level 6

The password for the next level is stored somewhere on the server and has all of the following properties: owned by user bandit7 owned by group bandit6 33 bytes in size

• Same deal as last level except different switches and searching from root.
• After running I also redirected stderr to /dev/null because there were too many permission denials getting in the way of reading output

bandit6@bandit:~$ find / -user bandit7 -group bandit6 -size 33c 2>/dev/null                                                 
/var/lib/dpkg/info/bandit7.password
bandit6@bandit:~$

• And we can cat it in a one liner for fun

bandit6@bandit:~$ cat $(find / -user bandit7 -group bandit6 -size 33c 2>/dev/null)                                          
morbNTDkSW6jIlUc0ymOdMaLnOlFVAaj
bandit6@bandit:~$

Level 7

The password for the next level is stored in the file data.txt next to the word millionth

• First instinct is to cat then pipe into grep for "millionth".

cat data.txt | grep millionth
millionth	dfwvzFQi4mU0wfNbFOe9RoWskMLg7eEc
bandit7@bandit:~$

• Yup

Level 8

The password for the next level is stored in the file data.txt and is the only line of text that occurs only once

• First instinct is to cat then pipe into uniq maybe.
• My first instinct, cat data.txt | uniq -u, contains too many entries. After researching I realize that repeated lines need to be adjacent to be detected as repeated, so let's sort then pipe into uniq.

bandit8@bandit:~$ sort data.txt | uniq -u                                                                                   
4CKMh1JI91bUIZZPXDqGanal4xvAg0JM
bandit8@bandit:~$

• Got it

Level 9

The password for the next level is stored in the file data.txt in one of the few human-readable strings, preceded by several ‘=’ characters.

• We can use strings to convert the garbage data to text, then grep the file for some "=".

bandit9@bandit:~$ strings data.txt -d  | grep ==
[========== the
T%========== passwordG
}========== ist"
========== FGUW5ilLVJrxX9kMYMmlN4MgbpfMiqey
bandit9@bandit:~$

• Password: FGUW5ilLVJrxX9kMYMmlN4MgbpfMiqey

Level 10

The password for the next level is stored in the file data.txt, which contains base64 encoded data

From wikipedia: >In computer programming, Base64 is a group of binary-to-text encoding schemes that transforms binary data into a sequence of printable characters, limited to a set of 64 unique characters. More specifically, the source binary data is taken 6 bits at a time, then this group of 6 bits is mapped to one of 64 unique characters.

• I can use base64 to decode the data by piping into it from cat.

bandit10@bandit:~$ cat data.txt | base64 -d 
The password is dtR173fZKb0RRsDFSGsg2RWnpNVj3qRr

• Success!

Level 11

The password for the next level is stored in the file data.txt, where all lowercase (a-z) and uppercase (A-Z) letters have been rotated by 13 positions

tr can be used to effectively rotate every character back.

bandit11@bandit:~$ cat data.txt | tr 'a-z' 'n-za-m' | tr 'A-Z' 'N-ZA-M'                                                     
The password is 7x16WNeHIi5YkIhWsfFIqoognUTyj9Q4

I need to learn tr syntax, I just found the implementation of Rot13 in tr online and added rotation for capitals as well and piped the output a second time. That worked.

Level 12

The password for the next level is stored in the file data.txt, which is a hexdump of a file that has been repeatedly compressed. For this level it may be useful to create a directory under /tmp in which you can work. Use mkdir with a hard to guess directory name. Or better, use the command “mktemp -d”. Then copy the datafile using cp, and rename it using mv (read the manpages!)

• mkdir /tmp/sm;cp data.txt /tmp/sm/; cd /tmp/sm
• I will probably have to convert from hex to binary or something and uncompress that multiple times
• xxd -r data.txt | gzip -d > out
• xxd can convert data from hex to binary and vice-versa.
• bzip2 out -d -c > out2
• gzip out2 -d -c > out3
• tar -xvf out3
• tar -xvf data5.bin
• bzip2 data6.bin -d -c > out7
• tar -xvf out7
• gzip data8.bin -d -c > out9

bandit12@bandit:/tmp/sm$ cat out9
The password is FO5dwFsc0cbaIiH0h8J2eUks2vdTDwAn
bandit12@bandit:/tmp/sm$

• I gave up on renaming every time so the scheme is inconsistent, but I just checked the output using file every time to determine what decompression algorithm or archival tool to use until it was completely unwrapped as a text file.
• Password: FO5dwFsc0cbaIiH0h8J2eUks2vdTDwAn

Level 13

The password for the next level is stored in /etc/bandit_pass/bandit14 and can only be read by user bandit14. For this level, you don’t get the next password, but you get a private SSH key that can be used to log into the next level. Note: localhost is a hostname that refers to the machine you are working on

bandit13@bandit:~$ cat sshkey.private 
-----BEGIN RSA PRIVATE KEY-----
MIIEpAIBAAKCAQEAxkkOE83W2cOT7IWhFc9aPaaQmQDdgzuXCv+ppZHa++buSkN+
gg0tcr7Fw8NLGa5+Uzec2rEg0WmeevB13AIoYp0MZyETq46t+jk9puNwZwIt9XgB
ZufGtZEwWbFWw/vVLNwOXBe4UWStGRWzgPpEeSv5Tb1VjLZIBdGphTIK22Amz6Zb
ThMsiMnyJafEwJ/T8PQO3myS91vUHEuoOMAzoUID4kN0MEZ3+XahyK0HJVq68KsV
ObefXG1vvA3GAJ29kxJaqvRfgYnqZryWN7w3CHjNU4c/2Jkp+n8L0SnxaNA+WYA7
jiPyTF0is8uzMlYQ4l1Lzh/8/MpvhCQF8r22dwIDAQABAoIBAQC6dWBjhyEOzjeA
J3j/RWmap9M5zfJ/wb2bfidNpwbB8rsJ4sZIDZQ7XuIh4LfygoAQSS+bBw3RXvzE
pvJt3SmU8hIDuLsCjL1VnBY5pY7Bju8g8aR/3FyjyNAqx/TLfzlLYfOu7i9Jet67
xAh0tONG/u8FB5I3LAI2Vp6OviwvdWeC4nOxCthldpuPKNLA8rmMMVRTKQ+7T2VS
nXmwYckKUcUgzoVSpiNZaS0zUDypdpy2+tRH3MQa5kqN1YKjvF8RC47woOYCktsD
o3FFpGNFec9Taa3Msy+DfQQhHKZFKIL3bJDONtmrVvtYK40/yeU4aZ/HA2DQzwhe
ol1AfiEhAoGBAOnVjosBkm7sblK+n4IEwPxs8sOmhPnTDUy5WGrpSCrXOmsVIBUf
laL3ZGLx3xCIwtCnEucB9DvN2HZkupc/h6hTKUYLqXuyLD8njTrbRhLgbC9QrKrS
M1F2fSTxVqPtZDlDMwjNR04xHA/fKh8bXXyTMqOHNJTHHNhbh3McdURjAoGBANkU
1hqfnw7+aXncJ9bjysr1ZWbqOE5Nd8AFgfwaKuGTTVX2NsUQnCMWdOp+wFak40JH
PKWkJNdBG+ex0H9JNQsTK3X5PBMAS8AfX0GrKeuwKWA6erytVTqjOfLYcdp5+z9s
8DtVCxDuVsM+i4X8UqIGOlvGbtKEVokHPFXP1q/dAoGAcHg5YX7WEehCgCYTzpO+
xysX8ScM2qS6xuZ3MqUWAxUWkh7NGZvhe0sGy9iOdANzwKw7mUUFViaCMR/t54W1
GC83sOs3D7n5Mj8x3NdO8xFit7dT9a245TvaoYQ7KgmqpSg/ScKCw4c3eiLava+J
3btnJeSIU+8ZXq9XjPRpKwUCgYA7z6LiOQKxNeXH3qHXcnHok855maUj5fJNpPbY
iDkyZ8ySF8GlcFsky8Yw6fWCqfG3zDrohJ5l9JmEsBh7SadkwsZhvecQcS9t4vby
9/8X4jS0P8ibfcKS4nBP+dT81kkkg5Z5MohXBORA7VWx+ACohcDEkprsQ+w32xeD
qT1EvQKBgQDKm8ws2ByvSUVs9GjTilCajFqLJ0eVYzRPaY6f++Gv/UVfAPV4c+S0
kAWpXbv5tbkkzbS0eaLPTKgLzavXtQoTtKwrjpolHKIHUz6Wu+n4abfAIRFubOdN
/+aLoRQ0yBDRbdXMsZN/jvY44eM+xRLdRVyMmdPtP8belRi2E2aEzA==
-----END RSA PRIVATE KEY-----

• I will copy this key to my host's ~/.ssh directory and login by forcing the use of the key for access: ssh bandit14@bandit.labs.overthewire.org -p 2220 -i .ssh/id_bandit14
• I'm in!

Level 14

The password for the next level can be retrieved by submitting the password of the current level to port 30000 on localhost.

• I can cat the password referenced in the last level and pipe it into netcat to send it to the port.

bandit14@bandit:~$ cat /etc/bandit_pass/bandit14 | netcat localhost  30000                                                  
Correct!
8xCjnmgoKbGLhHFAZlGE5Tmu4M2tKJQo

Level 15

The password for the next level can be retrieved by submitting the password of the current level to port 30001 on localhost using SSL encryption. Helpful note: Getting “HEARTBEATING” and “Read R BLOCK”? Use -ign_eof and read the “CONNECTED COMMANDS” section in the manpage. Next to ‘R’ and ‘Q’, the ‘B’ command also works in this version of that command…

• I will need to encrypt what I send to the port beforehand. This can be acchomplished with openssl's s_client.
• From the man page: >DESCRIPTION This command implements a generic SSL/TLS client which connects to a remote host using SSL/TLS. It is a very useful diagnostic tool for SSL servers.

openssl s_client -ign_eof -connect localhost:30001

• I then manually enter the password

8xCjnmgoKbGLhHFAZlGE5Tmu4M2tKJQo
Correct!
kSkvUpMQ7lBYyCM4GBPvCvT1BfWRy0Dx

closed
bandit15@bandit:~$

• Mission accomplished!

Level 16

The credentials for the next level can be retrieved by submitting the password of the current level to a port on localhost in the range 31000 to 32000. First find out which of these ports have a server listening on them. Then find out which of those speak SSL and which don’t. There is only 1 server that will give the next credentials, the others will simply send back to you whatever you send to it.

• I will search for candidate ports

bandit16@bandit:~$ netstat -tulpn
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
netstat: no support for `AF INET (tcp)' on this system.
tcp6       0      0 :::31691                :::*                    LISTEN      -                   
tcp6       0      0 :::31046                :::*                    LISTEN      -                   
tcp6       0      0 :::22                   :::*                    LISTEN      -                   
tcp6       0      0 :::2232                 :::*                    LISTEN      -                   
tcp6       0      0 :::2231                 :::*                    LISTEN      -                   
tcp6       0      0 :::2230                 :::*                    LISTEN      -                   
tcp6       0      0 :::2220                 :::*                    LISTEN      -                   
tcp6       0      0 :::31960                :::*                    LISTEN      -                   
bandit16@bandit:~$

• 31691, 31046, and 31960 are candidates
• However none are over ssl, the port may be open but not listening. I can scan with nmap.

nmap -p 31000-32000 localhost     
Starting Nmap 7.94SVN ( https://nmap.org ) at 2024-07-16 01:36 UTC
Nmap scan report for localhost (127.0.0.1)
Host is up (0.00014s latency).
Not shown: 996 closed tcp ports (conn-refused)
PORT      STATE SERVICE
31046/tcp open  unknown
31518/tcp open  unknown
31691/tcp open  unknown
31790/tcp open  unknown
31960/tcp open  unknown

Nmap done: 1 IP address (1 host up) scanned in 0.11 seconds

• I will not manually check each with s_client.

openssl s_client -ign_eof -connect localhost:31790 

kSkvUpMQ7lBYyCM4GBPvCvT1BfWRy0Dx
Correct!
-----BEGIN RSA PRIVATE KEY-----
MIIEogIBAAKCAQEAvmOkuifmMg6HL2YPIOjon6iWfbp7c3jx34YkYWqUH57SUdyJ
imZzeyGC0gtZPGujUSxiJSWI/oTqexh+cAMTSMlOJf7+BrJObArnxd9Y7YT2bRPQ
Ja6Lzb558YW3FZl87ORiO+rW4LCDCNd2lUvLE/GL2GWyuKN0K5iCd5TbtJzEkQTu
DSt2mcNn4rhAL+JFr56o4T6z8WWAW18BR6yGrMq7Q/kALHYW3OekePQAzL0VUYbW
JGTi65CxbCnzc/w4+mqQyvmzpWtMAzJTzAzQxNbkR2MBGySxDLrjg0LWN6sK7wNX
x0YVztz/zbIkPjfkU1jHS+9EbVNj+D1XFOJuaQIDAQABAoIBABagpxpM1aoLWfvD
KHcj10nqcoBc4oE11aFYQwik7xfW+24pRNuDE6SFthOar69jp5RlLwD1NhPx3iBl
J9nOM8OJ0VToum43UOS8YxF8WwhXriYGnc1sskbwpXOUDc9uX4+UESzH22P29ovd
d8WErY0gPxun8pbJLmxkAtWNhpMvfe0050vk9TL5wqbu9AlbssgTcCXkMQnPw9nC
YNN6DDP2lbcBrvgT9YCNL6C+ZKufD52yOQ9qOkwFTEQpjtF4uNtJom+asvlpmS8A
vLY9r60wYSvmZhNqBUrj7lyCtXMIu1kkd4w7F77k+DjHoAXyxcUp1DGL51sOmama
+TOWWgECgYEA8JtPxP0GRJ+IQkX262jM3dEIkza8ky5moIwUqYdsx0NxHgRRhORT
8c8hAuRBb2G82so8vUHk/fur85OEfc9TncnCY2crpoqsghifKLxrLgtT+qDpfZnx
SatLdt8GfQ85yA7hnWWJ2MxF3NaeSDm75Lsm+tBbAiyc9P2jGRNtMSkCgYEAypHd
HCctNi/FwjulhttFx/rHYKhLidZDFYeiE/v45bN4yFm8x7R/b0iE7KaszX+Exdvt
SghaTdcG0Knyw1bpJVyusavPzpaJMjdJ6tcFhVAbAjm7enCIvGCSx+X3l5SiWg0A
R57hJglezIiVjv3aGwHwvlZvtszK6zV6oXFAu0ECgYAbjo46T4hyP5tJi93V5HDi
Ttiek7xRVxUl+iU7rWkGAXFpMLFteQEsRr7PJ/lemmEY5eTDAFMLy9FL2m9oQWCg
R8VdwSk8r9FGLS+9aKcV5PI/WEKlwgXinB3OhYimtiG2Cg5JCqIZFHxD6MjEGOiu
L8ktHMPvodBwNsSBULpG0QKBgBAplTfC1HOnWiMGOU3KPwYWt0O6CdTkmJOmL8Ni
blh9elyZ9FsGxsgtRBXRsqXuz7wtsQAgLHxbdLq/ZJQ7YfzOKU4ZxEnabvXnvWkU
YOdjHdSOoKvDQNWu6ucyLRAWFuISeXw9a/9p7ftpxm0TSgyvmfLF2MIAEwyzRqaM
77pBAoGAMmjmIJdjp+Ez8duyn3ieo36yrttF5NSsJLAbxFpdlc1gvtGCWW+9Cq0b
dxviW8+TFVEBl1O4f7HVm6EpTscdDxU+bCXWkfjuRb7Dy9GOtt9JPsX8MBTakzh3
vBgsyi/sN3RqRBcGU40fOoZyfAMT8s1m/uYv52O6IgeuZ/ujbjY=
-----END RSA PRIVATE KEY-----

closed
bandit16@bandit:~$

• It was port 31790 and a key was returned, I will add it to my host's ~/.ssh. ssh bandit17@bandit.labs.overthewire.org -p 2220 -i ~/.ssh/id_bandit17 works!

Level 17

There are 2 files in the homedirectory: passwords.old and passwords.new. The password for the next level is in passwords.new and is the only line that has been changed between passwords.old and passwords.new NOTE: if you have solved this level and see ‘Byebye!’ when trying to log into bandit18, this is related to the next level, bandit19

• We can just diff to find the differing line:

bandit17@bandit:~$ diff passwords.new passwords.old 
42c42
< x2gLTTjFwMOhQ8oWNbMN362QKxfRqGlO
---
> FtePUTiLiwPzjIFw2T7o57oBS4zUvPpg

So the password is: x2gLTTjFwMOhQ8oWNbMN362QKxfRqGlO

Level 18

The password for the next level is stored in a file readme in the homedirectory. Unfortunately, someone has modified .bashrc to log you out when you log in with SSH.

• I know that you can pass a command to ssh to be ran rather than initializing a shell
• ssh bandit18@bandit.labs.overthewire.org -p 2220 'cat readme'
• The password was returned:

bandit18@bandit.labs.overthewire.org's password: cGWpMaKXVwDUNgPAVJbWYuGHVn9zl3j8

Level 19

To gain access to the next level, you should use the setuid binary in the homedirectory. Execute it without arguments to find out how to use it. The password for this level can be found in the usual place (/etc/bandit_pass), after you have used the setuid binary.

bandit19@bandit:~$ ./bandit20-do 
Run a command as another user.
  Example: ./bandit20-do id

• Evidently, we can use this binary to cat bandit20's password since it is the owner of the file that contains it.

bandit19@bandit:~$ ./bandit20-do cat /etc/bandit_pass/bandit20                                                              
0qXahG8ZjOVMN9Ghs7iOWsCfZyXOUbYO

Level 20

There is a setuid binary in the homedirectory that does the following: it makes a connection to localhost on the port you specify as a commandline argument. It then reads a line of text from the connection and compares it to the password in the previous level (bandit20). If the password is correct, it will transmit the password for the next level (bandit21). NOTE: Try connecting to your own network daemon to see if it works as you think

• I believe I have to connect to a port of my choosing using the suconnect binary and then send the password to that port while it is open. I have some options. I can either use a multiplexer or throw suconnect's process to the background.
• I am mistaken, it it the other way around. I must send the password to the port first, then suconnect grabs it.

bandit20@bandit:~$ echo -n "0qXahG8ZjOVMN9Ghs7iOWsCfZyXOUbYO" | netcat -l -p 30003 &                                     
[1] 867141
bandit20@bandit:~$ ./suconnect 30003                                                     
Read: 0qXahG8ZjOVMN9Ghs7iOWsCfZyXOUbYO
Password matches, sending next password
EeoULMCra2q0dSkYj561DX7s1CpBuOBt
[1]+  Done                    echo -n "0qXahG8ZjOVMN9Ghs7iOWsCfZyXOUbYO" | netcat -l -p 30003

• Password: EeoULMCra2q0dSkYj561DX7s1CpBuOBt

Level 21

A program is running automatically at regular intervals from cron, the time-based job scheduler. Look in /etc/cron.d/ for the configuration and see what command is being executed.

bandit21@bandit:~$ cat /etc/cron.d/cronjob_bandit22
@reboot bandit22 /usr/bin/cronjob_bandit22.sh &> /dev/null
* * * * * bandit22 /usr/bin/cronjob_bandit22.sh &> /dev/null
bandit21@bandit:~$ cat /usr/bin/cronjob_bandit22.sh
#!/bin/bash
chmod 644 /tmp/t7O6lds9S0RqQh9aMcz6ShpAoZKF7fgv
cat /etc/bandit_pass/bandit22 > /tmp/t7O6lds9S0RqQh9aMcz6ShpAoZKF7fgv
bandit21@bandit:~$ cat /tmp/t7O6lds9S0RqQh9aMcz6ShpAoZKF7fgv
tRae0UfB9v0UzbCdn9cY0gQnds9GF58Q
bandit21@bandit:~$

Password: tRae0UfB9v0UzbCdn9cY0gQnds9GF58Q

Level 22

A program is running automatically at regular intervals from cron, the time-based job scheduler. Look in /etc/cron.d/ for the configuration and see what command is being executed. NOTE: Looking at shell scripts written by other people is a very useful skill. The script for this level is intentionally made easy to read. If you are having problems understanding what it does, try executing it to see the debug information it prints.

bandit22@bandit:~$ cat /etc/cron.d/cronjob_bandit23
@reboot bandit23 /usr/bin/cronjob_bandit23.sh  &> /dev/null
* * * * * bandit23 /usr/bin/cronjob_bandit23.sh  &> /dev/null
bandit22@bandit:~$ cat /usr/bin/cronjob_bandit23.sh
#!/bin/bash

myname=$(whoami)
mytarget=$(echo I am user $myname | md5sum | cut -d ' ' -f 1)

echo "Copying passwordfile /etc/bandit_pass/$myname to /tmp/$mytarget"

cat /etc/bandit_pass/$myname > /tmp/$mytarget
bandit22@bandit:~$

• Based on this, the password whose name was hashed by md5sum
• I will manually run the command using "bandit23" as myname in a subshell and cat the appropriate file.

bandit22@bandit:~$ cat /tmp/$(echo I am user bandit23 | md5sum | cut -d ' ' -f 1)
0Zf11ioIjMVN551jX3CmStKLYqjk54Ga

Level 23

A program is running automatically at regular intervals from cron, the time-based job scheduler. Look in /etc/cron.d/ for the configuration and see what command is being executed. NOTE: This level requires you to create your own first shell-script. This is a very big step and you should be proud of yourself when you beat this level! NOTE 2: Keep in mind that your shell script is removed once executed, so you may want to keep a copy around…

bandit23@bandit:~$ cat /etc/cron.d/cronjob_bandit24 
@reboot bandit24 /usr/bin/cronjob_bandit24.sh &> /dev/null
* * * * * bandit24 /usr/bin/cronjob_bandit24.sh &> /dev/null
bandit23@bandit:~$ cat /usr/bin/cronjob_bandit24.sh
#!/bin/bash

myname=$(whoami)

cd /var/spool/$myname/foo
echo "Executing and deleting all scripts in /var/spool/$myname/foo:"
for i in * .*;
do
    if [ "$i" != "." -a "$i" != ".." ];
    then
        echo "Handling $i"
        owner="$(stat --format "%U" ./$i)"
        if [ "${owner}" = "bandit23" ]; then
            timeout -s 9 60 ./$i
        fi
        rm -f ./$i
    fi
done

• Based on the script that is being run by cron, we just need to write a script that gets the name of bandit24's password for us. This script executes every script in /var/spool/bandit24/foo/ if it is owned by bandit23 then deletes it. However, the script will be ran as bandit24 since it is that user's cronjob. Therefore, I can get the contents of /etc/bandit_pass from a script in /var/spool/bandit24/foo/.
• I am having an issue getting output from the script presumably due to permissions. I modified the target directory with chmod 777.

#!/bin/bash
cat /etc/bandit_pass/bandit24 >> /tmp/sm2/out.txt

This did the trick: gb8KRRCsshuZXI0tUuR6ypOFjiZbf3G8

• Success!

Level 24

A daemon is listening on port 30002 and will give you the password for bandit25 if given the password for bandit24 and a secret numeric 4-digit pincode. There is no way to retrieve the pincode except by going through all of the 10000 combinations, called brute-forcing. You do not need to create new connections each time

• This is the script I've formulated:

#!/bin/sh

code=0
while [ $code -le 9998 ];
do
        code=$((code+1))
        pass="gb8KRRCsshuZXI0tUuR6ypOFjiZbf3G8 ${code}"

        echo "Trying code $pass"
        echo $pass | nc localhost 30002 -w 1 | tee -a out.txt
done

• But it is very slow, there must be a faster way.
• Firstly, it starts at 0 because it just increments it doesn't handle every digit independently.

#!/bin/bash

for i in {0000..9999}
do
  pass="gb8KRRCsshuZXI0tUuR6ypOFjiZbf3G8 ${i}"
  echo "Trying code $pass"
  echo $pass | nc localhost 30002 -w 1 | tee -a out.txt
done

• This is better! Didn't know bash supported iterating over ranges this way. Now looking into improving speed. Could utilize parallelization in some way. I could:
  • Throw the jobs into the background with a max number running any time
  • Utilize parallel

#!/bin/bash

export pass_prefix="gb8KRRCsshuZXI0tUuR6ypOFjiZbf3G8 "

generate_and_try() {
  i=$1
  pass="${pass_prefix}${i}"
  echo "Trying code $pass"
  echo $pass | nc localhost 30002 -w 1 | tee -a out.txt
}

export -f generate_and_try
export pass_prefix

# Generate the sequence and run in parallel
seq -w 0000 9999 | parallel -j 100 generate_and_try

• parallel handles the parallelization... luckily it is installed...
• Instead of a loop this solution utilizes seq to generate codes
• The code is passed as a cmdline arg
• I will then search the output for the string "Correct". One could also grep -v the string "Wrong".

Correct! The password of user bandit25 is iCi86ttT4KSNe1armKiwbQNmB3YJP3q4

• Success!

Level 25

Logging in to bandit26 from bandit25 should be fairly easy… The shell for user bandit26 is not /bin/bash, but something else. Find out what it is, how it works and how to break out of it.

• I know that default shells are in /etc/passwd.
• cat /etc/passwd | grep -i bandit26 yields bandit26:x:11026:11026:bandit level 26:/home/bandit26:/usr/bin/showtext
• showtext is a script:

#!/bin/sh

export TERM=linux

exec more ~/text.txt exit 0

* And there is a key in the home directory 
```terminal
ls                                                                                                       
bandit26.sshkey
bandit25@bandit:~$ cat bandit26.sshkey 
-----BEGIN RSA PRIVATE KEY-----
MIIEpQIBAAKCAQEApis2AuoooEqeYWamtwX2k5z9uU1Afl2F8VyXQqbv/LTrIwdW
pTfaeRHXzr0Y0a5Oe3GB/+W2+PReif+bPZlzTY1XFwpk+DiHk1kmL0moEW8HJuT9
/5XbnpjSzn0eEAfFax2OcopjrzVqdBJQerkj0puv3UXY07AskgkyD5XepwGAlJOG
xZsMq1oZqQ0W29aBtfykuGie2bxroRjuAPrYM4o3MMmtlNE5fC4G9Ihq0eq73MDi
1ze6d2jIGce873qxn308BA2qhRPJNEbnPev5gI+5tU+UxebW8KLbk0EhoXB953Ix
3lgOIrT9Y6skRjsMSFmC6WN/O7ovu8QzGqxdywIDAQABAoIBAAaXoETtVT9GtpHW
qLaKHgYtLEO1tOFOhInWyolyZgL4inuRRva3CIvVEWK6TcnDyIlNL4MfcerehwGi
il4fQFvLR7E6UFcopvhJiSJHIcvPQ9FfNFR3dYcNOQ/IFvE73bEqMwSISPwiel6w
e1DjF3C7jHaS1s9PJfWFN982aublL/yLbJP+ou3ifdljS7QzjWZA8NRiMwmBGPIh
Yq8weR3jIVQl3ndEYxO7Cr/wXXebZwlP6CPZb67rBy0jg+366mxQbDZIwZYEaUME
zY5izFclr/kKj4s7NTRkC76Yx+rTNP5+BX+JT+rgz5aoQq8ghMw43NYwxjXym/MX
c8X8g0ECgYEA1crBUAR1gSkM+5mGjjoFLJKrFP+IhUHFh25qGI4Dcxxh1f3M53le
wF1rkp5SJnHRFm9IW3gM1JoF0PQxI5aXHRGHphwPeKnsQ/xQBRWCeYpqTme9amJV
tD3aDHkpIhYxkNxqol5gDCAt6tdFSxqPaNfdfsfaAOXiKGrQESUjIBcCgYEAxvmI
2ROJsBXaiM4Iyg9hUpjZIn8TW2UlH76pojFG6/KBd1NcnW3fu0ZUU790wAu7QbbU
i7pieeqCqSYcZsmkhnOvbdx54A6NNCR2btc+si6pDOe1jdsGdXISDRHFb9QxjZCj
6xzWMNvb5n1yUb9w9nfN1PZzATfUsOV+Fy8CbG0CgYEAifkTLwfhqZyLk2huTSWm
pzB0ltWfDpj22MNqVzR3h3d+sHLeJVjPzIe9396rF8KGdNsWsGlWpnJMZKDjgZsz
JQBmMc6UMYRARVP1dIKANN4eY0FSHfEebHcqXLho0mXOUTXe37DWfZza5V9Oify3
JquBd8uUptW1Ue41H4t/ErsCgYEArc5FYtF1QXIlfcDz3oUGz16itUZpgzlb71nd
1cbTm8EupCwWR5I1j+IEQU+JTUQyI1nwWcnKwZI+5kBbKNJUu/mLsRyY/UXYxEZh
ibrNklm94373kV1US/0DlZUDcQba7jz9Yp/C3dT/RlwoIw5mP3UxQCizFspNKOSe
euPeaxUCgYEAntklXwBbokgdDup/u/3ms5Lb/bm22zDOCg2HrlWQCqKEkWkAO6R5
/Wwyqhp/wTl8VXjxWo+W+DmewGdPHGQQ5fFdqgpuQpGUq24YZS8m66v5ANBwd76t
IZdtF5HXs2S5CADTwniUS5mX1HO9l5gUkk+h0cH5JnPtsMCnAUM+BRY=
-----END RSA PRIVATE KEY-----
bandit25@bandit:~$

• The solution for this was ultimately simple yet very elusive. So the text contained in ~text.txt was too short to do anything with and would immeditaly exit. I explored for any possible ssh switch or techqiue I could employ. I checked every permission. But the simpliest solution was just... to make the window smaller... That forces more to enter an interactive mode since all the text can't be displayed. From there you cna evert Vi and do whatever you need to.
• I first redefined the shell:
• :set shell=/bin/bash
• And enter a shell from Vi using:shell
• Now I am in and technically in:

Level 26

Good job getting a shell! Now hurry and grab the password for bandit27!

• Then catted the password

  bandit26@bandit:~$ cat /etc/bandit_pass/bandit26
  s0773xxkk0MXfdqOfPRVr9L3jJBUOgCZ
  

bandit26@bandit:~$ ```

• Password: s0773xxkk0MXfdqOfPRVr9L3jJBUOgCZ
• Remember this is just bandit 26's password.

bandit26@bandit:~$ ls -l
total 20
-rwsr-x--- 1 bandit27 bandit26 14880 Jul 17 15:57 bandit27-do
-rw-r----- 1 bandit26 bandit26   258 Jul 17 15:57 text.txt
bandit26@bandit:~$ ./bandit27-do cat /etc/bandit_pass/bandit27
upsNCc7vzaRDx6oZC6GiR6ERwe1MowGB
bandit26@bandit:~$

• Looks like there is a binary with the suid bit set allowing me to run it as bandit27 since they are the owner and bandit26's group has execute permissions on. The binary evidently takes a command as an argument to be ran. I used this to get the password for bandit27.
• Password: upsNCc7vzaRDx6oZC6GiR6ERwe1MowGB

Level 27

There is a git repository at ssh://bandit27-git@localhost/home/bandit27-git/repo via the port 2220. The password for the user bandit27-git is the same as for the user bandit27.

Clone the repository and find the password for the next level.

• I make a directory in tmp and clone the repo

bandit27@bandit:/tmp/gh$ git clone ssh://bandit27-git@localhost:2220/home/bandit27-git/repo

bandit27@bandit:/tmp/gh/repo$ cat README 
The password to the next level is: Yz9IpL0sBcCeuG7m9uQFt8ZNpS4HZRcN

• There was simply a README containing the password

Level 28

There is a git repository at ssh://bandit28-git@localhost/home/bandit28-git/repo via the port 2220. The password for the user bandit28-git is the same as for the user bandit28. Clone the repository and find the password for the next level.

• It is the same process as before, but the contents are little different.

bandit28@bandit:/tmp/gh28/repo$ ls
README.md
bandit28@bandit:/tmp/gh28/repo$ cat README.md 
# Bandit Notes
Some notes for level29 of bandit.

## credentials

- username: bandit29
- password: xxxxxxxxxx

bandit28@bandit:/tmp/gh28/repo$

• Looks like we will have to brute for a password a 10 character password, which would take way too long. As in like millions of years... using a supercomputer... so this can't be the solution.

bandit28@bandit:/tmp/gh28/repo$ git log
WARNING: terminal is not fully functional
Press RETURN to continue 
commit 8cbd1e08d1879415541ba19ddee3579e80e3f61a (HEAD -> mast
er, origin/master, origin/HEAD)
Author: Morla Porla <morla@overthewire.org>
Date:   Wed Jul 17 15:57:30 2024 +0000

    fix info leak

commit 73f5d0435070c8922da12177dc93f40b2285e22a
Author: Morla Porla <morla@overthewire.org>
Date:   Wed Jul 17 15:57:30 2024 +0000

    add missing data

commit 5f7265568c7b503b276ec20f677b68c92b43b712
Author: Ben Dover <noone@overthewire.org>
Date:   Wed Jul 17 15:57:30 2024 +0000

    initial commit of README.md
bandit28@bandit:/tmp/gh28/repo$

• The solution looks it is going to be looking at each commit to see when a password added. This can be a accomplished with checkout. There seems to be three commits and the password was ultimately in the middle one.

bandit28@bandit:/tmp/gh28/repo$ git checkout 73f5d0435070c8922da12177dc93f40b2285e22a
Previous HEAD position was 5f72655 initial commit of README.md
HEAD is now at 73f5d04 add missing data
bandit28@bandit:/tmp/gh28/repo$ ls
README.md
bandit28@bandit:/tmp/gh28/repo$ cat README.md 
# Bandit Notes
Some notes for level29 of bandit.

## credentials

- username: bandit29
- password: 4pT1t5DENaYuqnqvadYs1oE4QLCdjmJ7

bandit28@bandit:/tmp/gh28/repo$ 

• Password: 4pT1t5DENaYuqnqvadYs1oE4QLCdjmJ7

Level 29

There is a git repository at ssh://bandit29-git@localhost/home/bandit29-git/repo via the port 2220. The password for the user bandit29-git is the same as for the user bandit29. Clone the repository and find the password for the next level.

bandit29@bandit:/tmp/b29/repo$ cat README.md 
# Bandit Notes
Some notes for bandit30 of bandit.

## credentials

- username: bandit30
- password: <no passwords in production!>

bandit29@bandit:/tmp/b29/repo$

• Probably going to have to go to a different branch based on this.

bandit29@bandit:/tmp/b29/repo$ git branch -r
WARNING: terminal is not fully functional
Press RETURN to continue 
  origin/HEAD -> origin/master
  origin/dev
  origin/master
  origin/sploits-dev
bandit29@bandit:/tmp/b29/repo$

• I'll checkout dev

bandit29@bandit:/tmp/b29/repo$ git checkout dev
Previous HEAD position was 5a53eb8 initial commit of README.md
branch 'dev' set up to track 'origin/dev'.
Switched to a new branch 'dev'
bandit29@bandit:/tmp/b29/repo$ ls
code  README.md
bandit29@bandit:/tmp/b29/repo$ ls code/
gif2ascii.py
bandit29@bandit:/tmp/b29/repo$ cat README.md 
# Bandit Notes
Some notes for bandit30 of bandit.

## credentials

- username: bandit30
- password: qp30ex3VLz5MDG1n91YowTv4Q8l7CDZL

bandit29@bandit:/tmp/b29/repo$

• There it is! Password: qp30ex3VLz5MDG1n91YowTv4Q8l7CDZL

Level 30

There is a git repository at ssh://bandit30-git@localhost/home/bandit30-git/repo via the port 2220. The password for the user bandit30-git is the same as for the user bandit30. Clone the repository and find the password for the next level.

bandit30@bandit:/tmp/b30/repo$ cat README.md 
just an epmty file... muahaha
bandit30@bandit:/tmp/b30/repo$ git tag
WARNING: terminal is not fully functional
Press RETURN to continue 
secret
bandit30@bandit:/tmp/b30/repo$ git show secret
WARNING: terminal is not fully functional
Press RETURN to continue 
fb5S2xb7bRyFmAvQYQGEqsbhVyJqhnDy
bandit30@bandit:/tmp/b30/repo$

• The README had nothing, but there was a tag named secret that contained the password.
• Password: fb5S2xb7bRyFmAvQYQGEqsbhVyJqhnDy

Level 31

There is a git repository at ssh://bandit31-git@localhost/home/bandit31-git/repo via the port 2220. The password for the user bandit31-git is the same as for the user bandit31. Clone the repository and find the password for the next level.

bandit31@bandit:/tmp/b31/repo$ cat README.md 
This time your task is to push a file to the remote repository.

Details:
    File name: key.txt
    Content: 'May I come in?'
    Branch: master

bandit31@bandit:/tmp/b31/repo$

• Here it seems we must write a file key.txt that contains the required string.
• We can touch key.txt; echo "May I come in? > key.txt"
• Then add key.txt, commit, then push.
• The .gitignore needed to be edited. because every text file was being ignored via: *.txt
• Once pushed we succeeded!

bandit31-git@localhost's password: 
Enumerating objects: 4, done.
Counting objects: 100% (4/4), done.
Delta compression using up to 2 threads
Compressing objects: 100% (2/2), done.
Writing objects: 100% (3/3), 321 bytes | 321.00 KiB/s, done.
Total 3 (delta 0), reused 0 (delta 0), pack-reused 0
remote: ### Attempting to validate files... ####
remote: 
remote: .oOo.oOo.oOo.oOo.oOo.oOo.oOo.oOo.oOo.oOo.
remote: 
remote: Well done! Here is the password for the next level:
remote: 3O9RfhqyAlVBEZpVb6LYStshZoqoSx5K 
remote: 
remote: .oOo.oOo.oOo.oOo.oOo.oOo.oOo.oOo.oOo.oOo.
remote: 
To ssh://localhost:2220/home/bandit31-git/repo
 ! [remote rejected] master -> master (pre-receive hook declined)
error: failed to push some refs to 'ssh://localhost:2220/home/bandit31-git/repo'
bandit31@bandit:/tmp/b31/repo$

• Password: 3O9RfhqyAlVBEZpVb6LYStshZoqoSx5K

Level 32

After all this git stuff, it’s time for another escape. Good luck!

WELCOME TO THE UPPERCASE SHELL
>> ls
sh: 1: LS: Permission denied

Where am I?

• We can't use commands because they are all made uppercase. In this peculiar situation we'd have access to environment variables and command line arguments however. Considering that the first arg, $0, is the shell itself, i can call a new shell with it?

>> $0
$ 

• YES!
• I switched to a bash shell and got the next password

bandit33@bandit:~$ cat /etc/bandit_pass/bandit33
tQdtbs5D5i2vJwkO8mEyYEyTL8izoeJ0
bandit33@bandit:~$ 

Level 33

At this moment, level 34 does not exist yet.

bandit33@bandit:~$ cat README.txt 
Congratulations on solving the last level of this game!

At this moment, there are no more levels to play in this game. However, we are constantly working
on new levels and will most likely expand this game with more levels soon.
Keep an eye out for an announcement on our usual communication channels!
In the meantime, you could play some of our other wargames.
us
If you have an idea for an awesome new level, please let us know!
bandit33@bandit:~$

I WON! 🥳

What I Learned

This game was very fun and forces us to really think about how certain aspects of a Linux system works and how they interact with one another. The topics that are generally touched on are:

• Basic command line utilities
  • ls, cd, mkdir, etc.
• Text manipulation
  • cat, grep, uniq, sort, seq, strings, tr
• Redirection
  • Redirection Operators, Pipes, tee
• Permissions
• Remote Access and Remote Command Execution
  • ssh
• System File Locations and Root Directories
  • etc/passwd, /tmp, `/bin
• Processes
  • &, bg, fg, tmux, screen, ps
• Scheduling
  • cron
• Shells
  • bash, sh
  • Environment Variables and Arguments, $0, TERM
• Base Conversions, Encoding, and Hashing
  • base64, md5sum
• Compression and Archiving
  • tar, gzip, bzip2
• Basic networking Concepts and Utilities
  • nc, netstat
• Scripting
  • Conditionals, Comparisons, Loops
  • Optionally parallelization through parallel
• Git Concepts and Subcommands
  • checkout, branch, tag, add, commit, push

Basic Calculus in C (2024-07-10)

Numerical Methods of Differentiation

I just wrote this up because I was curious about how to code calculus stuff which I had never done before. This lead me to falling down a rabbit hole on numerical methods.

Anyone who has taken Calculus in school is familiar with some numerical methods of differentiation or integration. The basis of the definition of a derivative is itself an approximation:

\[ f'(x) = \lim_{{h \to 0}} \frac{f(x+h) - f(x)}{h} \]

A secant line where the distance between the two intersections is an infinitesimal.

I've recently learned that this is a class of method known as the Finite Difference Method where a derivative is calculated using the same difference quotient with a finite distance that is sufficiently small:

\[f'(x) \approx \frac{f(x+h) - f(x)}{h}\]

New to me is the Symmetric Difference Quotient which also approximates the slope, but by using two secant lines equidistant from x.

\[f'(x) \approx \frac{f(x+h) - f(x-h)}{2h}\]

The accuracy will be off by a proportion of h using the former method and by h^2 using the latter one.

The accuracy will also be affected by the choice of forward, back, or central difference, round-off error, or truncation error, which funnily enough is often notated using big-O.

Numerical Methods of Integration

Riemann Sums is the other numerical method that is often learned to introduce the concept of an integral. And has been in use since antiquity - like when Archimedes approximated the area of the circle through methods of exhaustion.

Riemann sums partition the area under a curve either from the left, right, or midpoint and if those partitions or slices are sufficiently small, you get a good approximation. This method utilizes rectangles:

\[\int_{a}^{b} f(x) , dx \approx \sum_{i=0}^{n-1} f\left( \frac{x_i + x_{i+1}}{2} \right) \Delta x\]

But can utilize other polygons such a trapezoids:

\[\int_{a}^{b} f(x) , dx \approx \frac{\Delta x}{2} \left[ f(x_0) + 2 \sum_{i=1}^{n-1} f(x_i) + f(x_n) \right]\]

There are a plethora numerical methods for both differentiation and integration that have been developed over the centuries that have their applications in engineering, manufacturing, physics, and computer science.

Numerical Methods in C

Since these methods discretize functions they are how one could create algorithms to perform differentiation or integration. For example:

// Forward difference method
double forward_difference(double x, double h) {
    return (f(x + h) - f(x)) / h;
}

// Backward difference method
double backward_difference(double x, double h) {
    return (f(x) - f(x - h)) / h;
}

// Central difference method
double central_difference(double x, double h) {
    return (f(x + h) - f(x - h)) / (2 * h);
}

These c function will approximate a derivative given some function.

How about integration:

// Trapezoidal rule function
double trapezoidal(double a, double b, int n) {
    double h = (b - a) / n;
    double sum = 0.5 * (f(a) + f(b));
    
    for (int i = 1; i < n; i++) {
        sum += f(a + i * h);
    }
    
    return sum * h;
}

This will calculate an integral given a number of sub-intervals.

Or this:

// Simpson's rule function
double simpsons(double a, double b, int n) {
    double h = (b - a) / n;
    double sum = f(a) + f(b);
    
    for (int i = 1; i < n; i++) {
        double x = a + i * h;
        if (i % 2 == 0) {
            sum += 2 * f(x);
        } else {
            sum += 4 * f(x);
        }
    }
    
    return sum * h / 3;
}

Which does the same according to Simpson's Rule.

In Conclusion

Numerical methods seem essential to most calculus applications in the real world, however it isn't emphasized at all in the university calculus series. I'm glad I read up on it. I may update this with more advanced methods translated to c.

Rewrites of GNU Utilities (2024-06-27)

There seems to be a modern trend of rewriting common GNU utilities, especially in Rust. The GNU utils have been rewritten before, most notably as BusyBox, the GNU core utils as a single binary meant to be used in extremely small embedded Linux environments. However, Rust seems to be a popular language used to write these newer ones, evidently because Rust is a new darling language. However it seems to have a newfound relevance as rust code was recently added to the Linux kernel.
These rewrites often offer improved functionality over their much older counterparts, have more human readable options, syntax, and outputs, and some even boast greater efficiency and speed. Now, if true, these improvements are likely never human detectable. No person will likely ever feel a difference in speed at a cli, but it is fun to note. Despite these improvements these rewrites will likely never gain widespread use these the originals are already installed and countless scripts and other tools already depend on their existence. But a poweruser who wants to have fun and utilize newer, more powerful and intuitive tools, should have at it. I'd actually recommend many of them, they're amazing. I'd just caution these users to not come too accustomed to using them as if they ever want to become a professional, they'll immediately encounter a situation where only GNU utils are at their disposal. I definitely have to keep this in mind when either using a rewrite or my own aliases.
In this post, I'm going list the ones I personally use and others I've never tried but am interested in.

bat

A cat clone with syntax highlighting and Git integration.

exa

A modern replacement for ls.

lsd

The next gen file listing command. Backwards compatible with ls.

delta

A viewer for git and diff output

dust

A more intuitive version of du written in rust.

duf

A better df alternative

broot

A new way to see and navigate directory trees

fd

A simple, fast and user-friendly alternative to find.

ripgrep

An extremely fast alternative to grep that respects your gitignore

ag

A code searching tool similar to ack, but faster.

fzf

A general purpose command-line fuzzy finder.

mcfly

Fly through your shell history. Great Scott!

choose

A human-friendly and fast alternative to cut and (sometimes) awk

jq

sed for JSON data.

sd

An intuitive find & replace CLI (sed alternative).

cheat

Create and view interactive cheatsheets on the command-line.

tldr

A community effort to simplify man pages with practical examples.

bottom

Yet another cross-platform graphical process/system monitor.

glances

Glances an Eye on your system. A top/htop alternative for GNU/Linux, BSD, Mac OS and Windows operating systems.

gtop

System monitoring dashboard for terminal.

hyperfine

A command-line benchmarking tool.

gping

ping, but with a graph.

procs

A modern replacement for ps written in Rust.

httpie

A modern, user-friendly command-line HTTP client for the API era.

curlie

The power of curl, the ease of use of httpie.

xh

A friendly and fast tool for sending HTTP requests. It reimplements as much as possible of HTTPie's excellent design, with a focus on improved performance.

zoxide

A smarter cd command inspired by z.

dog

A user-friendly command-line DNS client. dig on steroids

Linux Learning Resources (2024-06-26)

I am currently a RHCSA who will eventually be an RHCE. This requires me to keep my skills fresh. The learning seemingly never stops; there is always something else to know. Whether it be a new tool that is really cool, or a facet of something's design that is intriguing or indicates how it can be manipulated.
All this constant learning and re-learning requires resources. I've read an untold number of documentation/wiki articles, and have watched countless Youtube videos on topics relating to Linux. Here, I want to document the resources that have served me best and that have provided the most digestible knowledge. Maybe it will help others. It will be categorized along the lines of format. Also, each resource will be rated from 1-5 penguins 🐧. The more penguins the better.

Books

I find books the worst method of learning Linux. It isn't conducive to hands on learning. However, portions of these books so relay and explain underlying concepts pretty well. Most books take the form of that of a manual that is good for referencing but not learning. And even then, something printed on paper easily becomes outdated. Books that are meant for preparing for a specific exam are better as they often include lab material.

• Linux Bible by Christopher Negus, published on August 5, 2020. 🐧🐧🐧

• Unix and Linux System Administration Handbook by Evi Nemeth, Garth Snyder, Trent R. Hein, Ben Whaley, and Dan Mackin, published on July 14, 2017.🐧🐧🐧

• Red Hat RHCSA 9 Cert Guide: Red Hat Enterprise Linux 9 (EX200) by Sander van Vugt, published on January 7, 2023. 🐧🐧🐧🐧

• Linux From Scratch by Gerard Beekmans, published on August 1, 2021. 🐧🐧

• ProLUG Big Book of Labs 🐧🐧🐧🐧🐧

  • ProLUG Book Git Repo

• Linux Kernel in a Nutshell by Greg Kroah-Hartman, published on December 22, 2006. 🐧🐧

• Linux Kernel Development by Robert Love, published on October 12, 2010. 🐧🐧

Wikis

In my opinion, wikis are the best written resource. They often succinctly describe the nature of something and provide the most pertinent tools and their use cases. They are also written and maintained by the experts themselves. Examples are often provided as snippets that can be copied and pasted directly into a terminal.

• ArchWiki 🐧🐧🐧🐧

• Gentoo Wiki 🐧🐧🐧

• Red Hat Customer Portal 🐧🐧🐧🐧🐧

• Debian Wiki 🐧

Websites

There are websites out their specifically dedicated to learning Linux. Some are small readings that include a quizzes for example. Others are hands on labs similar to something like HackTheBox. Some are just there to help guide continuing education.

• OverTheWire(Bandit) 🐧🐧🐧🐧🐧
• Linux Journey 🐧🐧🐧🐧🐧
• DevOps Learning Roadmaps 🐧🐧🐧🐧🐧
• Kubernetes Learning Roadmaps 🐧🐧🐧🐧🐧

Youtube Channels

The issue with Youtube channels relating to Linux is that they are often not related to administration at all. They are mostly from the perspective of a desktop user, albeit a poweruser. Some may lean towards admin more than others, and some may focus entirely on scripting. But most focus on the industry and landscape in general. Regardless, these are the channels I've watched and enjoyed over the years, and have learned from greatly.

• Chris Titus Tech 🐧🐧🐧🐧🐧
• Brodie Robertson🐧🐧🐧
• DistoTube🐧🐧🐧🐧
• Linux Cast🐧🐧

Why mdbook? (2024-06-24)

There are a plethora of static site generation tools out there. One could essentially use any language or framework they desire. My requirements were a tool that is visually and functionally simple. I am not a front end developer, and I have no desire to be one. My first attempt was using a Jekyll template. While Jekyll and it's templates seem decent, they didn't offer the amount of editing capabilities I was looking for. Rather than being locked into a directory structure or toolbar formatting. I've also considered tools like Docusaurus because every website I've seen in it looks nice, but it seems I'd be directly editing source more than I'd like to. Eventually, since I have experience with writing in Markdown, especially with Gitbook and mdbook, I decided to use what I am most familiar with.

Why Markdown is Amazing

Markdown is probably the best text formatting "language" other than Latex. It is simple and portable. It's parsed and ultimately converted into HTML so it can exist essentially anywhere. And tools like Pandoc allow one to convert into into any document format one needs like pdf or even the dreaded troff for man pages. Aside form conversions numerous platforms support markdown natively like Github and the zettelkasten-astic note taking program Obsidian. One could write Markdown in one, and take it essentially anywhere. Notes, webpages, and documentation oh my. And if you need functionality the specific parser you're using doesn't support, many support overriding the defaults because, everything will ultimately be HTML and CSS. You could resort to editing anything you needed to by hand anyway.

mdbook As A Personal Site

The rust🦀 project, mdbook is definitely intended to be used for books, not personal sites. However the chapter bar is functionally a navbar,each, I can still link to other pages from within a page if I have to, and it naturally lends itself to a blog. Now the cons are that mdbook lacks a tagging system. Or any system for automatically maintaining blog posts. However, like I said, I'm not locked into any structure. I can invent any scheme my heart desires through the use of shell scripts and Github actions. I can design a system that assigns names according to a convention, fills in content for posts according to a template, orders the posts by date written, implement some kind os psuedo tagging system, etc. All that being said, I was always looking for something quick and dirt, where I can just market myself and flex my interests. This will do.

Links

• The Markdown Guide
• The Markdown Guide Book They Deserve The Money
• This site's repo

Notes

This chapter is where I will dump my notes. Anything is fair game. I use Obsidian to take notes, which uses Markdown, so I can simply copy/paste whatever I write to here.

Cheatsheet DUMP

Bash Cheatsheet

Redirection Cheatsheet

Important System Directories and Files

/etc

/var

/usr

/boot

/proc

/sys

/dev

Vim Cheatsheet

tmux Cheatsheet

tmux Hotkeys Cheatsheet

SSH Cheatsheet

Git Cheatsheet

Systemd Cheatsheet

Docker Cheatsheet

Kubernetes Cheatsheet

Ansible Cheatsheet

iptables Cheatsheet

nftables Cheatsheet

Cron Cheatsheet

awk Cheatsheet

sed Cheatsheet

grep Cheatsheet

find Cheatsheet

Screen Cheatsheet

rsync Cheatsheet

tar Cheatsheet

gzip Cheatsheet

bzip2 Cheatsheet

curl Cheatsheet

wget Cheatsheet

MySQL/MariaDB Cheatsheet

PostgreSQL Cheatsheet

Nginx Cheatsheet

Apache Cheatsheet

Firewalld Cheatsheet

UFW Cheatsheet

SELinux Cheatsheet

AppArmor Cheatsheet

LDAP Cheatsheet

LVM Cheatsheet

ZFS Cheatsheet

Btrfs Cheatsheet

Network Troubleshooting Cheatsheet

System Diagnostics and Logging Cheatsheet

dmesg