Part III of Building a Web Server Awesomely: Set up DDNS to deal with your dynamic IP – for free!

Part III of Building a Web Server Awesomely: Set up DDNS to deal with your dynamic IP – for free!

   December 22, 2017       Unknown

---

This is the third part in a series of how to turn a Raspberry Pi into a fully-functional Web server, serving multiple custom domains. Want more context? Read the introductory post.

---

OK. So far, we have the great wide Web talking to your router by referencing its IP directly. That's cool, and it works. For now. However, most of us have Internet Service Providers (ISPs) that don't guarantee us a static IP — or if they do, we pay prettily for it. Any day now, your IP may change without warning, and your domain will still be pointing to your old IP address. It is tragic, but we will overcome this.

The way we get around this is by using a Dynamic DNS service (DDNS). This service will keep track of your IP and either:

1. Update your domain's DNS records to point to the new IP, or;
2. Provide you with your own subdomain — for example, whatareheirloomtomatoes.freeddns.org — that gets updated to point to the new IP.

Why would you do the latter when you could do the former? Dollars. To have a DDNS service update your domain's DNS records directly, you will need a premium service like DynDNS. So if we want to do this economy-style, what we use a free service that provides the latter and change our domain's DNS records to point to that always-updated subdomain. Essentially, your domain points to the DDNS subdomain, which then points to your router's IP.

But how does the DDNS service know when your IP has changed? Again, one of two ways:

1. You configure your router to connect directly to the DDNS service and keep it posted about any changes.
2. You run a little cronjob, provided by the DDNS service, on your server's machine to periodically check your IP address and let the DDNS service know if it's changed.

Again, (1) definitely seems easier, so I know you're immediately suspicious. I've trained you well. That functionality is only available on some routers, and they will only connect to the top, big-name DDNS services like DynDNS and no-ip. Plus, (2) isn't so bad. You just download the software on your machine and configure it once — set it and forget it.

Here's what I recommend:

1. Create an account with Dynu, a free DDNS service.
2. Create a single free subdomain, again like whatareheirloomtomatoes.freeddns.org.
3. Set up the cron job to keep your IP up-to-date. Dynu has full instructions for Pi users.

Great, you've set up your DDNS service. Now let's change your domain to use it.

1. Open up your domain's DNS settings.
2. Remove the 'A' record we created earlier, the one that points to your IP. That's so old school.
3. Create a new CNAME record that points to your DDNS subdomain. The name should be www; this indicates that it's the www subdomain of your domain that points to the DDNS subdomain. (More on that in a moment.) It should look something like www | CNAME | 1h | whatareheirloomtomatoes.freeddns.org
4. You can't create a CNAME record for the root of your domain ('@') like we did for the 'A' record. So as it stands, this will only work properly if people type in wwwwhatareheirloomtomatoes.com, and not without the www prefix. To fix this, we add a synthetic record to the domain that will forward the root to the www subdomain. Look in your DNS settings for subdomain forwarding, and add a record like @ | whatareheirloomtomatoes.com | www.whatareheirloomtomatoes.com; i.e., forward the root ('@') to the www URL.

There it is — you should now be impervious to IP address changes. I can confirm this setup worked for me recently when I installed a new router; all my websites were back up in a minutes without any intervention, other than configuring the router as explained in Part I (i.e. giving the Pi a static internal IP, port forwarding).

Now, if you do want to use no-ip so that your router will automatically update your DDNS service on IP change (and you've checked that your router supports this), you can do that too. Sign up for a free account and do everything as above — just be aware that you will have to acknowledge that you want to keep your account every month so they don't close it.

---

Now that everything is perfect and resilient with your Web server, next up is setting up multiple Web servers on your Pi and using the reverse proxy to send traffic to the correct one. Coming soon!

Read More

Part II of Building a Web Server Awesomely: Host a single server (custom domain edition)

Part II of Building a Web Server Awesomely: Host a single server (custom domain edition)

   December 17, 2017       Unknown

---

This is the second part in a series of how to turn a Raspberry Pi into a fully-functional Web server, serving multiple custom domains. Want more context? Read the introductory post.

---

OK. In Part I, we set up our Pi and our router. Now, if you hit up your router's IP address from the wide Web, traffic will be passed through to your Pi.

Unfortunately, right now, your Pi is dropping that on the floor.

But we can fix that. It's time to install nodejs on your Pi.

Installing node

1. Open up your SSH client (ex. Bitvise) and ssh into your Pi. The host value is the IP of your Pi; this is the static internal IP we reserved in Part I. Your username is the one you created in Part I. Use the default port 22 and log in. Enter the password for the account that you created in Part I.
2. If you're using Bitvise, it should have opened an SFTP window and an SSH window. For now, focus on the SSH window (the terminal).
3. Follow these instructions to install node on your Pi, then come back.

Create your node project

1. Make yourself a directory to host your site. I created a web directory, and within that, one for my first site, e.g. whatareheirloomtomatoes
2. In that leaf directory, initialize your node project with npm init. Fill in the configuration values you want, or leave the defaults.
3. Put some stuff in there that you want to serve up. Create a public sub-directory; now you can throw in some html pages, client-side javascript, css -- whatever you want. You can test it out by creating an index.html file with <html><body>Hello world</body></html> in it.

Run your node project

Install http-server

To start, we're going to use a node package that can serve up your static files with a simple, premade http server. Later, we can customize this if we want to write our own server code (which, honestly, is where node shines). But let's get this working end-to-end first.

We want to be able to use this package globally, since it's not project-specific, so we use the -g flag:

sudo npm install http-server -g

Check out the documentation if you're curious how it works or what the parameters are.

Run it!

Try running your new server. Within your main directory (ex. whatareheirloomtomatoes), run:

http-server

It defaults to serving up the public sub-directory and the port 8080. So let's navigate to:

http://<your-pi-ip>:8080

Again, the IP is the one we reserved with your router and logged into the Pi with. You should now see your Hello World page!

However, the other great folks on the Web can't. The above address is your internal IP, so you can only hit it from machines on your LAN. If others try putting in your router's IP address, they're getting warmer, but we still have a disconnect: Your Pi doesn't know to pass through the requests it receives on its port 80 to your new node server running on 8080.

Letting the world see your beautiful site

Everyone should get to see the amazing Hello World page that you made. Everyone. So we're going to make sure that requests to port 80 on your Pi get forwarded to your Web server running on 8080.

We're going to use nginx as a reverse proxy. It will be listening for requests on port 80, sent from your router; then it will proxy the requests to your node server running on 8080.

Follow these instructions to install nginx, up until they mention php; then panic, close that tab, and come back to this one, where we are all javascript, all the time.

Then follow these instructions to set up the reverse proxy configuration ("Configure nginx reverse proxy" section only).

Once you've configured it all and reloaded your nginx config, try hitting your external IP address (i.e. your router's IP) from your browser. (Don't know your external IP? Google "what's my IP address.") Hopefully, it shows you the same thing as when you hit your Pi's local IP and port.

Pointing your domain to your Web server

We could still stand to make this easier for people to get to, and you didn't spend $12 on a custom domain for nothing. Let's configure that fancy domain to point to your external IP.

1. Log in to your domain's DNS management. I used Google Domains, but this should work with any provider.
2. Add a custom resource record. We want an 'A' record. (Want to know why? Read about it.) The name value should be @ (the root). You can leave the TTL at 60; the value of the data field should be your router's IP address. It should look something like: @ | a | 60 | 12.345.678.90
3. Save that biz.
4. It might take a bit to propagate, but not long. In a few minutes, try accessing your domain. You should see the same thing as when you type in your fully-qualified external IP address.

Making a custom Web server, not this http-server nonsense

If you just wanted to serve up a static site, you're golden. Keep dropping stuff in your public folder and go to town. If, however, you wanted to write some of your own server code, this is your chance.

I'm going to leave most of this as an exercise you, dear reader, since there are many guides out there to help, but I'll leave some hints here:

1. Express: The http module is low-level and would force you to write a lot of boilerplate. Express abstracts that away. Install it in each of your projects.
2. Nodemon: This listens for changes in your server and dynamically rebuilds, so it's quick to iterate and test changes. Instead of running node server.js, you'd run nodemon server.js. As with http-server, globally install this one with -g.
3. Forever: It keeps your server running... forever. It will restart after failures and background your task so it doesn't die when you close your terminal, etc. Instead of running node server.js, you'd run forever start server.js. Install this one globally, too.
4. Crontab time: I recommend adding a forever command to start your server to your crontab. That way, if your power goes out, or your Pi otherwise reboots, so will your server. Something like this: @reboot /usr/local/bin/forever start /home/<username>/web/whatareheirloomtomatoes/server.js >> /home/<username>/web/whatareheirloomtomatoes/server.out 2>&1
5. The power of SFTP: Write your code on your local machine, not your Pi. I write it on my Windows box using, as I mentioned, Atom. I then upload it with my SFTP client, Bitvise, onto my Pi. It is good and wonderful and I never need to look at my Pi's GUI ever.

---

Now that you have your beautiful, publicly-available Web server, we will next make it resilient to router IP address changes in Part III: Set up DDNS to deal with your dynamic IP – for free!

Read More

Part I of Building a Web Server Awesomely: Set up your Pi, router, & development machine

Part I of Building a Web Server Awesomely: Set up your Pi, router, & development machine

   September 25, 2017       Unknown

---

This is the first part in a series of how to turn a Raspberry Pi into a fully-functional Web server, serving multiple custom domains. Want more context? Read the introductory post.

---

Let us begin. Here's what you'll need:

• A raspberry pi.
• A router. Bet you already have one of these, since you're on this site.
• A development machine.

I know it said that all in the title, but I have no sense of what people do or do not read.

Pi time!

You're going to want a Pi, an SD card, a heat sink, and probably a case because that's cool. You can get all of these things in a starter kit. Mine's in a clear case so I can watch it merrily blink away.

Get it all set up per the instructions, all the way through logging in and starting up the GUI, and then come back to me.

Hello! Good job booting up your Pi! A couple important things to note:

• You should add your own account to the Pi that you'll use for most of your development. If not, you must change the default password of the default account (pi/raspberry). Now the most important part in either case: You must set your locale properly before you choose a password for your account. Either just trust me, or read this to understand better.
• I bet you think it's all fun and games to have your Pi all wireless. No. Plug it into your router with a wired ethernet connection. This will save you an hour later when you're wondering why it's so slow to ssh into your Pi. Plus, we're not clowns. We want the response time for our server to be baller.

Let's get into that router config

There will come a moment in Part II when you've got your Web server set up and think you're a god because you can reach it from the computer it's running on – perhaps even from another computer on your LAN. You are not this thing, not unless you configure your router. Your computer is on a LAN, my friend, and the outside world can't access a random port on a random machine on your LAN. Your Pi doesn't even have its own external IP address. Internal, sure. And we'll get to know it quite well. But only your router has an external IP address. Therefore, logically, your router is going to have to be the gatekeeper when people are trying to get at your Web server.

So we're going to do two things:

1. Give your Pi a static internal IP address.
2. Forward traffic to port 80 on your router to your Pi.

Static internal IP

This is my service to you: I give you tips that, while not strictly necessary, will save you much heartache. Sometimes you'll need to reboot your Pi, and when you do, your router may not necessarily assign it the same internal IP address. And yet so much of your life has come to care about this IP address: Your SSH client, for example, and your router, when it comes to forwarding traffic.

So first, go to 192.168.1.1 in your browser. This address is your router's IP. Your other machines will have similar addresses, differing by the integers after the last dot. Log in to your router. (If you don't know the default username and password, Google your router brand or check here. Most are admin/admin or admin/password.)

Next, probably somewhere in advanced settings, you'll see something about DHCP or LAN setup. Mine, for Netgear, was under Advanced > LAN setup > Address reservation. Choose a low-ish integer to assign your Pi; don't choose 1, because if you were reading carefully, that is reserved for your router; don't choose a number in use by another machine, because that will be confusing for you for 5 minutes. More in-depth reading here.

Forwarding port 80

This is so when a machine on the internet knocks on your router's door at the default HTTP port, 80, it knows to send that traffic to your Pi's port 80, where your Web server will be eagerly listening.

In my router, this configuration is hidden under Advanced > Advanced settings > Port forwarding, because it is double advanced. Add an HTTP service that forwards from external port 80 to port 80 at the static, internal IP address you just assigned your Pi in the step above.

Your development setup

You'll probably want another computer on which you can install an IDE and ssh into the Pi. It's not strictly necessary if you have a lightweight IDE for your Pi (read: emacs, vim) and a screen to connect it to, but personally I prefer to ssh.

The particulars are a matter of personal preference, but you'll want an IDE, an SSH client, and an SFTP client. I'm using:

• Atom: IDE
• Bitvise: SFTP & SSH client

Atom is neat because it's a highly configurable IDE with a lot of community support, so it's got tons of plugins; I installed some to support Javascript, node, Github, etc. But mostly prettification. Who can read CSS that's not indented properly?

My development flow is to edit local files with Atom and regularly upload them to the Pi using Bitvise. I use Bitvise's shell to do my SSHing, so starting up servers, editing my crontab, aggressively refreshing configurations – all the good stuff.

---

That, my friends, is the end of the "setting up basic stuff" post. Next up is actually setting up our first Web server on the Pi. If you're ready for more, proceed to Part II.

Read More

I built a personal Web server*, and I did it awesomely

I built a personal Web server*, and I did it awesomely

   September 15, 2017       Unknown

---

*JK I built 5.

A month or so ago, I was sitting on the floor of my apartment, feeling a sense of ennui. I needed a new hobby. Lying on trigger point therapy balls while reading sci fi, while satisfying, wasn't enough anymore. I was itching to make something.

It just so happens that I'd had a fun idea for a Web application I'd like to build, about which I will say no more because it's not done and you can't have my idea. Around the same time, I got caught up by friends' wedding excitement and offered to build and host some wedding websites.

It doesn't sound like you'd need your own Web server for that, you're thinking. No, it doesn't. But I wanted one.

It's borderline irresponsible to host your own Web server, what with cloud this and cloud that, but I'm feeling irresponsible. And I'm feeling like I want to play with new things.

Thus was born this adventure. Come along with me. Build a Web server awesomely.

---

I'm going to lay out the ultimate state, just to get you real excited. If it doesn't make sense yet, don't panic; we'll go through each step below.

We're going to make it so that you can have a single Raspberry Pi hosting multiple Web servers, each serving traffic to a different domain name (or subdomain). We'll do this by having all of these domains point to our router's (dynamic, external) IP address, which is configured to forward requests to port 80 to the (static, internal) IP address of our Raspberry Pi. On the Pi, we're running a a reverse proxy server that forwards these requests, based on the domain name in the request, to individual Web servers on other ports, each one serving a single domain. Except I lied there. We'll actually have the domains point to a DDNS hostname, which itself points to our router's IP; we do this because our ISP doesn't guarantee us a static IP address and it could change at any moment.

To make it concrete: You go to www.whatareheirloomtomatoes.com. Because my router has a dynamic IP address, I can't just have www.whatareheirloomtomatoes.com redirect to my router's IP (or I can, but when my router gets assigned a new IP out of nowhere, I'll be sad); instead, I have a CNAME record that aliases my domain to my DDNS hostname, which I've registered for with a DDNS service (e.g. no-ip.com or dynu.com). This DDNS service gets notified when my IP changes (by a small piece of software they provide to poll for changes), so it knows what my current IP is, and forwards your request for www.whatareheirloomtomatoes.com to that IP on port 80. My router sends that request to my Pi on port 80, where nginx is listening; nginx looks up www.whatareheirloomtomatoes.com in a configuration file I've set up and sees that it should forward that request to the port I've specified, say 8080. On 8080, I have a nodejs server running that serves up the content for www.whatareheirloomtomatoes.com. Ta-da!

If you only want to do a subset of these things – like only set up a single Web server, no reverse proxying, no DDNS madness – I've modularized the topics and you can call it quits when it suits you.

By the end of this, here are all of the technologies we will have played with:

• Development environment:
  • Atom: IDE (optional; use whatever IDE you want)
  • PuTTY: SSH client (optional; only if sshing from Windows to your pi)
  • Bitvise: SFTP client (optional; again only if sshing from Windows to your pi)
• Web server:
  • Nginx: Reverse proxying
  • Nodejs: Web servers (optional; if you want to serve a static site, you can use nginx alone. Or use Apache, if you want to party like it's 1999. I don't care.)
    • Forever: Keep those Web servers running
    • Nodemon: Reload your server on change
  • ddclient: Update your DDNS service when your ip changes (optional, if your router can connect directly to your DDNS service)
• Configuration you never cared to know about:
  • Port forwarding on your router
  • Domain DNS settings (for those who have a custom domain)

---

Ready to start? Follow the guides below on your path to awesomeness.

1. Part I: Setting up your enviroment: The Pi, the router, and your development machine
2. Part II: Host a single server (custom domain edition)
3. Part III: Set up DDNS to deal with your dynamic IP – for free!
4. Part IV: Why host one server when you could have n servers? [coming soon]

Read More