Category: Linux

Simple Linux (Ubuntu) Security
I’m setting up a Bitcoin full node to perform some analyses on the full blockchain. Here are quick steps I followed to improve the security of a basic Ubuntu box for these purposes [1]. This is not meant to be an exhaustive list! Rather, this should serve as a general approach that helps you think about how you can continue to improve and monitor the security of the systems you’re using. And, while I’m using Ubuntu, these general principles apply to other Linux distros, too.
Step 1: Use What You Know

When I was younger, I remember someone suggesting to my father that he could use a simpler tool to test a TV component instead of the oscilloscope he was already using. However, he promptly replied, “I like to use what I know best,” and he just kept using the oscilloscope. This wisdom applies to the security realm, too.

My first step towards configuring a full node involves ditching Windows. This shouldn’t be a interpreted as an indictment against the possible security of Windows. Rather, I don’t know Windows security as well as I know Linux security, and I should stick with what I know best. And, I chose Ubuntu because I’ve used this distro the most over the past year. [Note to future self: remember that for some reason burning DVDs at any of the faster speeds on my iMac leads to images that can’t be mounted 🙁 ]

Step 2: Run The Bare Necessities

Sure, with Ubuntu, you can have it all… but just know ALL includes possible vulnerabilities, too. When I install Ubuntu for this type of purpose, I first start by installing the “Minimum Install,” an option that some other distros offer, too. Once installed, I follow the steps below to remove more unneeded stuff (“stuff” is a technical term that is often under-appreciated, but not here, my friends, not here.)
1. Run the command below to view the running processes:
  # sudo lsof -i
2. Identify a process that seems unneeded (e.g., cups, avahi-daemon, etc.)
3. Remove the package using the command below [2]:
  # sudo apt-get autoremove cups
4. Restart the computer and make sure there are no errors.
5. Rinse and repeat until the song above plays like an anthem when I view the processes.
Step 3: Filter Communications

Angry and upset teenager screaming at the mobile telephone

I’ve learned to filter communications on social media with some people so I can avoid drama… not you, of course [note to self…]. In the same way, helping computers filter communications can help limit your system administration drama.

I’ve learned to love iptables, and so can you. As in any relationship, it first starts by listening… figuratively and literally. Often, when you run the command below, you’ll see that iptables is overly ACCEPTing in terms of communications (oh, Ubuntu, how quaint):
# sudo iptables -S

This is one situation in which it’s preferable NOT to be accepting. In fact, in order to prioritize security, we start by blocking everything, and then we explicitly accept what we absolutely need (e.g., port 80 for apt-get, port 53 for DNS, etc.) There are plenty of tutorials on iptables out there (e.g., here, here, and here), but hopefully these principles point you in the general direction so you can get started [3].

Steps 4 – ∞: Update & Monitor

Mother Hand holding video baby monitor for security of the baby

New parents know the joy of bringing home a new baby to the nursery that they’ve carefully prepared. Every little detail in the room has been painstakingly considered. New parents also quickly realize that the easy work was preparing the room. Every new day going forward requires constant updates and monitoring (e.g., new diapers, new clothes, new furniture, new diap… whoops, sorry, I was dreaming of sleep and forgot I already said… zzzzzzzzzz…)

In the same way, our precious new Ubuntu box will require constant updates and monitoring. If you study pen-testing techniques, you’ll quickly realize a common theme: exploits are old! Unless you’re up against an incredibly brilliant hacker or powerful nation state, the hackers attacking your system are searching for vulnerabilities in software that have already been patched. If you diligently update your system, you can avoid the vast majority of exploits that could be used against your baby. It’s simple:
# sudo apt-get update
# sudo apt-get upgrade

Finally, because you know your little baby better than anyone else, because you’ve limited your system to the bare necessities, because you’ve filtered your communications, and because you’ve diligently updated your system, you can sometimes identify zero-day exploits (and mitigate them) before there’s even a patch (all zero-days have to transition to one-days somehow, and you could be the one!) Monitoring the logs and noticing suspicious activity on a well-secured system is much easier because there is less to see: nefarious behavior that is new will stand out. Stay vigilant, my friends!

Final Step: I Give You… THE INTERNET!

You’re ready to let your little baby go out into the world [4]. Fair thee well!

All-Time Classic!

Footnotes
1. You will often hear of hardening a system to reduce the attack surface for a specific use. I hate this term, because a “hardened” system soon becomes soft and squishy if it’s not actively updated and monitored and improved. Remember, security is a process, not a product.
2. There are various ways to remove packages using apt-get. I use the autoremove option because it automatically removes associated dependencies, too, if they are otherwise unused.
3. Remember, rules are ephemeral unless you take steps to make them persistent. New users are often surprised to see their configuration work disappear when they next reboot. I like to use a combination of the iptables-persistent package and the iptables-save command to make my rules persistent.
4. There are certainly other things one can (and should) do, such as contain the possible damage apps/processes can do to the rest of the system, but I’ll cover these approaches in a later post.
February 25, 2021
Automatically Generating Content Inventories (Part 1)
Introduction

I’ll admit it, in my youth (say, a few days ago) I’d often generate a content inventory by hand. I’d simply open a new spreadsheet and start working my through the site until I was done chronicling the content. I chose this path because of its simplicity and because many of the websites I work on are quite small.

This month I’m working with a client on several sites, and the total number of pages is close to one thousand. Sure, I’ll likely still want to view each of the pages just in case the title and description fail to reflect the content (or it’s an asset that lacks this meta information), but automatically generating the url, file type, title and description should save a tremendous amount of time.

To automatically generate a content inventory, we’ll break the work up into three steps:
1. Create a local copy of the website (covered in this post.)
2. Create a list of broken links (covered in this post.)
3. Parse the local files to create a spreadsheet (covered in the next post.)
Using Wget To Create A Local Copy Of Your Website

The GNU wget package makes it very easy to generate a local copy of a website. You can use it to crawl your entire website and download all of the linked assets (html files, images, pdf’s, etc.) While you can install wget on Windows and Macs, when I’m using one of these systems I just run a VM of my favorite Linux distro, which already has wget installed. I found a great tutorial that demonstrates how to create a mirror of a website with wget, and it’s most basic usage is illustrated by the command below.
```
$ wget -m http://www.site.com/
```
There are many more options, but the command above would create the directory “www.site.com” and put all of the linked files from your website in that directory.

Using Wget To Find Broken Links (404)

Next, let’s make sure we have a list of the broken links in the website. After all, a content inventory is supposed to guide future work, and all future work should take into account content that’s either missing or unfindable.

Again, making use of wget greatly simplifies this task, and I found another great tutorial that outlines using wget to find broken links. The basic command structure is listed below.
```
$ wget --spider -o file.log -r -p http://www.site.com
```
Once completed, you have a file that you can grep / search for occurrences of 404 errors.

A Bash Script To Automate Simplify Things

Of course, I’m old and I forget things easily. I can’t be expected to remember these commands for the next five minutes, let alone the next time I’m creating a content inventory a month from now. Additionally, instead of using multiple calls to wget, we can merge these operations into one roundtrip. Here’s a simple bash script that automates the creation of the local mirror of the website and the log file with broken link information.
```
#!/bin/bash

# remember to run chmod +x myFileNameWhateverItIs

# store domain
echo "Enter website domain (e.g., www.site.com):"
read domain
# store url
url="http://$domain"
# system status
echo "Creating mirror..."
# create local mirror
wget -m -w 2 -o wget.log -p $url
# system status
echo "Creating broken link log..."
# store broken link(s) info
grep -n -B 2 '404 Not Found' wget.log > wget-404.log
# system status
echo "Process completed."
```
If I store the code above in the file “local-site.sh” (and call chmod +x on it), I can call it directly to create a local copy of the website and a log file containing broken links:
```
$ ./local-site.sh
> Enter website domain (e.g., www.site.com):
> www.example.com
> Creating mirror...
> Creating broken link log...
> Process completed.
```
I’ll cover parsing of the local files to create a content inventory spreadsheet in the next post.
September 1, 2014