How to connect to local development server from Internet?

So I'm working on a project where we need two-way communication with third-party services.
For instance, if we want to create webhook code for HubSpot that triggers when the form property cancellation_date is "set to less than 1 day ago" we would need to have a callback URL that gets called when an event occurs.

For starters, this callback URL should ideally be pointing to our local development machine, where code can be developed and tested. Once we are done working on and testing the webhook code, we can deploy it to some static server and set the webhook URL to hosted service.

What's on the market?

Multiple tools will create a tunnel for us; most of them are paid, subscription-based services, which we want to avoid as the last thing we need is another monthly fee to pay.

Custom subdomains mean the ability to specify a subdomain name, like mysubdomain.acme.com, where acme.com is the domain name.
Without a custom subdomain, our endpoint URL would randomly change each time we restart the tunnel.

Here are some services:

- Cloudflared
 Powerful, free service, which we will discuss in this article.
 
- LocalTunnel
  Completely free service that allows custom subdomains.
  It is not reliable as the server is often unavailable for prolonged periods.
  On the plus side, it can be self-hosted.

- NgRok
  Paid service, which in its free tear allows for only one tunnel and no  
  custom subdomains.

- LocalXpose
  Paid service, which in its free tear allows for 4 tunnels and  
  custom subdomains. There is a 15-minute timeout limit per tunnel 🤷‍♂️

Presenting  ✨Cloudflared✨

After trying many services, we have concluded that Cloudflared offers the best free service. It can be easy to set up and allows advanced configuration for pro users.

The first thing to do is install the cloudflared client on your machine.

I want it quick and dirty

Zero configuration, quick tunnel with a random subdomain name can be created by issuing the terminal command:

$ cloudflared tunnel --url http://localhost:3000

This will generate a random subdomain on trycloudflare.com that points to the local machine on port 3000.

$ cloudflared tunnel --url http://localhost:3000
2022-06-30T15:48:05Z INF Thank you for trying Cloudflare Tunnel. Doing so, without a Cloudflare account, is a quick way to experiment and try it out. However, be aware that these account-less Tunnels have no uptime guarantee. If you intend to use Tunnels in production you should use a pre-created named tunnel by following: https://developers.cloudflare.com/cloudflare-one/connections/connect-apps
2022-06-30T15:48:05Z INF Requesting new quick Tunnel on trycloudflare.com...
2022-06-30T15:48:08Z INF +--------------------------------------------------------------------------------------------+
2022-06-30T15:48:08Z INF |  Your quick Tunnel has been created! Visit it at (it may take some time to be reachable):  |
2022-06-30T15:48:08Z INF |  https://cp-allocation-folders-chargers.trycloudflare.com                                  |
2022-06-30T15:48:08Z INF +--------------------------------------------------------------------------------------------+
2022-06-30T15:48:08Z INF Cannot determine default configuration path. No file [config.yml config.yaml] in [~/.cloudflared ~/.cloudflare-warp ~/cloudflare-warp /etc/cloudflared /usr/local/etc/cloudflared]
2022-06-30T15:48:08Z INF Version 2022.6.3
2022-06-30T15:48:08Z INF GOOS: darwin, GOVersion: go1.18.3, GoArch: arm64
2022-06-30T15:48:08Z INF Settings: map[protocol:quic url:http://localhost:3000]
2022-06-30T15:48:08Z INF cloudflared will not automatically update if installed by a package manager.
2022-06-30T15:48:08Z INF Generated Connector ID: 4905a1a4-d6a6-41ad-9535-7c9b1abeec92
2022-06-30T15:48:08Z INF Initial protocol quic
2022-06-30T15:48:08Z INF Starting metrics server on 127.0.0.1:56817/metrics
2022-06-30T15:48:21Z INF Connection a35bd5d8-f02c-4243-af6f-7dd2ca473866 registered connIndex=0 ip=198.41.200.13 location=AMS
2022-06-30T15:48:22Z INF Connection 1aa4dabe-e6cc-467b-876b-3bc84fd6d6db registered connIndex=1 ip=198.41.192.227 location=ZAG
2022-06-30T15:48:22Z INF Connection 8a7df147-4367-4655-994f-9c9bdc381d27 registered connIndex=2 ip=198.41.200.193 location=AMS
2022-06-30T15:48:24Z INF Connection 60504070-98b5-4dd9-9073-67685618c7ae registered connIndex=3 ip=198.41.192.107 location=ZAG

What if I have multiple services hosted locally?

The story complicates a bit if you need to host multiple services to the Internet, like a NodeJS server and React frontends. Let's explore a scenario where we have a NodeJS server, customer frontend app, and administration frontend app that must be exposed to the Internet. We will need custom subdomains as we don't want a new URL each time we start our tunnel.

Let's buy a domain
The idea is to have custom subdomains under our domain managed through Cloudflare.
Any domain will suffice, and domains can be purchased for a few bucks a year on sites like Namecheap or GoDaddy. This is the only cost.

The purchased domain must be added to Cloudflare.

Setting things up through Cloudflared CLI

Once our new domain is added to Cloudflare, we can finish our process from the terminal.

We need to connect Cloudflared to our Cloudflare account and domain by issuing the command:

$ cloudflared tunnel login

The default browser should open where the domain should be selected and authorized.

If all goes fine, Cloudflared will create and store credentials file for future use.

If the browser failed to open, please visit the URL above directly in your browser.
You have successfully logged in.
If you wish to copy your credentials to a server, they have been saved to:
/Users/johndoe/.cloudflared/cert.pem

We are now ready to create the tunnel:

$ cloudflared tunnel create myTunnel

Which creates a new tunnel:

Tunnel credentials written to /Users/johndoe/.cloudflared/f61a648d-4d77-4849-bd7a-3826af11dbfb.json. cloudflared chose this file based on where your origin certificate was found. Keep this file secret. To revoke these credentials, delete the tunnel.

Created tunnel myTunnel with id f61a648d-4d77-4849-bd7a-3826af11dbfb

We now need to specify services in the /Users/johndoe/.cloudflared/config.yaml file.
Open the text editor and create the config.yaml file with an ingress configuration similar to the one below:

tunnel: f61a648d-4d77-4849-bd7a-3826af11dbfb
credentials-file: /Users/johndoe/.cloudflared/f61a648d-4d77-4849-bd7a-3826af11dbfb.json
ingress:
  - hostname: backend.mydevtunnel.us
    service: http://localhost:4000
  - hostname: frontend-admin.mydevtunnel.us
    service: http://localhost:3000
  - hostname: frontend.mydevtunnel.us
    service: http://localhost:3001
  - service: http_status:404

Assuming:
- there is a backend server running locally on port 4000 that we want to access trough https://backend.mydevtunnel.us
- there is a React frontend server hosting administration pages locally on port 3000 that we want to access trough https://frontend-admin.mydevtunnel.us
- there is a React frontend server hosting customer pages locally on port 3001 that we want to access trough https://frontend.mydevtunnel.us

The last step is to create Cloudflare DNS routes.

$ cloudflared tunnel route dns f61a648d-4d77-4849-bd7a-3826af11dbfb backend.mydevtunnel.us
$ cloudflared tunnel route dns f61a648d-4d77-4849-bd7a-3826af11dbfb frontend-admin.mydevtunnel.us
$ cloudflared tunnel route dns f61a648d-4d77-4849-bd7a-3826af11dbfb frontend.mydevtunnel.us

Which on success generates the output:

2022-06-30T16:41:36Z INF Added CNAME backend.mydevtunnel.us which will route to this tunnel tunnelID=f61a648d-4d77-4849-bd7a-3826af11dbfb
2022-06-30T16:41:37Z INF Added CNAME frontend-admin.mydevtunnel.us which will route to this tunnel tunnelID=f61a648d-4d77-4849-bd7a-3826af11dbfb
2022-06-30T16:41:37Z INF Added CNAME frontend.mydevtunnel.us which will route to this tunnel tunnelID=f61a648d-4d77-4849-bd7a-3826af11dbfb

We can see our routes on the Cloudflare dashboard.

And that's it! We can now fire the tunnel up with:

$ cloudflared tunnel run myTunnel

With output like:

2022-06-30T20:27:12Z INF Starting tunnel tunnelID=f61a648d-4d77-4849-bd7a-3826af11dbfb
2022-06-30T20:27:12Z INF Version 2022.6.3
2022-06-30T20:27:12Z INF GOOS: darwin, GOVersion: go1.18.3, GoArch: arm64
2022-06-30T20:27:12Z INF Settings: map[cred-file:/Users/johndoe/.cloudflared/f61a648d-4d77-4849-bd7a-3826af11dbfb .json credentials-file:/Users/johndoe/.cloudflared/f61a648d-4d77-4849-bd7a-3826af11dbfb.json]
2022-06-30T20:27:12Z INF Generated Connector ID: 48ba03d1-ec79-4cdf-a6cc-2c57a0ec11e5
2022-06-30T20:27:12Z INF cloudflared will not automatically update if installed by a package manager.
2022-06-30T20:27:12Z INF Initial protocol quic
2022-06-30T20:27:12Z INF Starting metrics server on 127.0.0.1:61036/metrics
2022-06-30T20:27:13Z ERR update check failed error="no release found"
2022-06-30T20:27:13Z INF Connection 1f8b75a5-e784-42b1-a0a2-ab6033974cb4 registered connIndex=0 ip=198.41.200.233 location=VIE
2022-06-30T20:27:14Z INF Connection 5265e214-7295-40b2-93b9-276a169433c7 registered connIndex=1 ip=198.41.192.167 location=ZAG
2022-06-30T20:27:15Z INF Connection f952b7bf-412a-445d-a03c-e6d2cd907674 registered connIndex=2 ip=198.41.200.193 location=AMS
2022-06-30T20:27:16Z INF Connection 093773e4-9ebc-43ab-9df3-3b3990a839a4 registered connIndex=3 ip=198.41.192.47 location=ZAG

To test the thing working, we can spawn a simple HTTP server with the command:

$ python3 -m http.server --cgi 4000

The backend URL is pointing to our mock server.

Before we wrap up

Cloudflare is caching content by default. This can be problematic for static content like compiled and minified HTML, JS, and CSS files.

If we want to have a non-cached version of our great React frontend we would need to set it up in the Cloudflare dashboard.

It can be done in two ways:

1. Enable development mode

2. Create an individual page rule
Sometimes we want only specific pages not to be cached.