Local development with Subdomains, Mobile Testing, and OAuth

tl;dr

For many applications, especially when starting out, localhost has all the functionality needed. As applications get more complex, there are many options including tunneling and remote development environments, but localhost with a local DNS can provide another viable option.

Full Length

While working on mazaar.io, I needed a way to interact with the app while working on the changes. The most straightforward way of doing this is typically using localhost, which is how I started, but as my app grew in complexity, so did my requirements.

First, Mazaar supports custom subdomains, so I needed to handle subdomains. I didn't know it at the time, but modern browsers now support subdomains on localhost.

Then, I realized 90% of my user sessions were on mobile, so I wanted to be able to easily interact with my working changes from my phone. Dev tools does a great first pass while developing, but it's not the same feedback as actually interacting with the design on a mobile device.

Next, I added Facebook and Google logins because I have seen this massively improve signup conversion rate for other companies. Google and Facebook have different requirements for their callback URLS, but to accommodate both you'll need a URL that is public and served over HTTPS.

With these requirements, I landed on a solution that involved using a Cloudflare Tunnel on a domain that would send traffic to my machine. This has worked really well for the last several months, but there was a small cost associated with it. Over the last few weeks, I've been focused on cutting costs wherever I can, including migrating off Heroku; so, as part of that, I decided to reevaluate my approach.

Goal: Save money and simplify the approach.

Here's a summary of the options I considered and their capabilities, below I will add more detail for each option.

localhost

As I mentioned earlier, I learned through this process that modern browsers now support subdomains on localhost (that definitely ages me 😂), so localhost would actually be sufficient for many applications. For my purposes, localhost on it's own can support interacting with my working changes from a separate device, like my phone.

If it were not for subdomains, I could connect to my running server through the IP address or my computer's hostname, but once you need subdomains, this option is not viable.

Loopback Domains (i.e. lvh.me)

There are a few domains out there, like lvh.me, that simply point back to 127.0.0.1 (aka localhost). I've used domains like this in the past to workaround localhost not supporting subdomains, but with that change, you can see from my table that there's isn't really any advantage to using them today.

• You can't connect from another device since the domain will point back to the device that is making the request.
• You likely can't oauth because it won't be an SSL connection.
• It's not private, you're trusting the domain to handle your traffic politely. lvh.me has in fact not pointed to 127.0.0.1 briefly in the past.

ngrok

ngrok is a popular tool for creating a tunnel from the public internet to your machine. Basically, you run a program, known as an agent, on your machine that connects to ngrok's network and gives you a random URL in their network (like blah.ngrok-free.app). When a request hits ngrok's network at your random URL, it's forwarded to your machine via the agent.

This approach allows us to connect other devices to the running application which includes mobile devices and webhooks.

The downside of this approach is the random URL changes every time you start the tunnel which will result in some complexity in your code. All of your traffic is passing through ngrok's network, so you are trusting them with your traffic; but, I personally feel better about doing this with a company offering this service than the loopback domains. Lastly, internet is of course required for this approach since it utilizes the public internet.

This approach does allow testing your oauth providers, but since it's a random domain, you'll have to update your configs each time.

ngrok with custom domain

We can alleviate most of the pain from using ngrok by paying the $8/user/month to use a custom domain. This allows us to reduce complexity in the code and test oauth without updating our callback urls as much.

The main downside here, is that $8/user/month could add up for teams with several developers.

Cloudflare Tunnels (+ ACM)

Cloudflare is another provider of tunneling. The upside of Cloudflare is that you may already be using it for DNS, in which case it allows consolidation of vendors.

It has nearly the same capabilities as ngrok with the main distinction that I needed to subscribe to their Advanced Certificate Manager to get an HTTPS connection to my localhost. This brings the cost for Cloudflare to $10/mo plus the cost of your domain renewal; so for some teams, this could be a cheaper solution than ngrok.

localhost, .test, and DNSMasq

Like I mentioned, I used Cloudflare Tunnels with ACM for a while and it worked well, but in an effort to cut that $10/mo and simplify my stack, I ended up with an approach using localhost, .test, and DNSMasq.

So for starters, I configured my application to run on mazaar.localhost in dev and got everything working. There weren't any big hurdles in doing this, it mostly involved changes to my development.rb file.

Then I started working on allowing my phone to connect to the running server. This is when I stumbled upon the idea of using DNS to accomplish the objective. The idea is actually pretty simple, my computer and phone are on the same network, so I just need my network to be smart enough to send requests for a given domain to my computer.

If I had a fancy router with DNS built in, I could set this up there, but instead I can accomplish this goal in a few steps:

• First, we need to setup my computer to be a DNS. To do this, we'll leverage DNSMasq.
• Then, we'll configure DNSMasq to point all traffic for a given domain to my computer's IP address on my network.
• We'll update our application server to bind to port 0.0.0.0 so that all traffic is allowed.
• Lastly, we update my phone's DNS settings to use my computer as a DNS instead of the router.

And voila, now my phone can interact with my running application on my computer. I like this approach because it's free, doesn't require external vendors, and doesn't require internet. I think it's really neat that it leverages existing network behavior to accomplish my goals.

The main downside is that this wouldn't work for oauth or webhooks. It's also a little annoying to edit the network settings in your phone.

For oauth, I could use ngrok when needed to test things, but since oauth is already setup for Mazaar, it's actually more important to test this in prod anyway. For webhooks, it looks like Stripe's CLI might work with this to accomplish what I need and there are other options if I need to test webhooks from multiple vendors. Regardless, I'd prefer to opt-in to tunneling when needed than for it to be the default.

Notes

• I ended up using mazaar.test for development because from my reading my understanding is that localhost should always point to 127.0.0.1 and my DNS changes point the domain to the IP address of my computer on my network.
• I'm running my Capybara tests on mazaar.localhost because some behavior broke when it was running on mazaar.test, specifically Omniauth mocking broke.
• I updated the settings in my router to keep my IP assigned to my computer to avoid needing to update the DNS file.

Remote Development

One option I didn't try is doing all of your development remotely in something like Github Workspaces. From what it looks like, I think this would provide all the functionality needed except, you'd be dependent on internet and be locked into their pricing. I've worked in this kind of environment in the past and it worked well, so it might be a good option for some.

Conclusion

Now did I accomplish my goal...somewhat. I'm definitely saving the money, but I'm not sure I'd call the solution simpler. Additionally, I've lost some capabilities from the previous approach. For me personally, I'm happy with these compromises, but I could certainly see the argument for the other approaches.

References

• Hacker News Discussion on TLDs in Development
• IANA Special Use Domains