Design a minimalist URL shortening service with AWS Lambda and seamlessly integrated with GitHub as the storage
A. Architecture overview:
What we gonna to build today is a simple shorten URLs service like: bit.ly, Facebook, Twitter shorten URLs service, ...
Instead of using the a very long url like: example.com/this-is-very-long-long-long-long-long-url, we can use something like: example.com/my-url
How it works:
We will create a JSON file on Github that contains our shorten urls as keys and our original link as values.
And when users make request to our Lambda function using shorten URLs, we will retrieve the original links from Github and redirect users to the original links.
B. Tutorial
Step 1: Download Python Serverless template.
Run the following command to get the template:
# Clone the template, make sure there is no folder named gh-short-url before:
git clone -b without_serverless_account https://github.com/sexydevops/fastapi-aws-starter-kit.git gh-short-url
# Change directory to the template:
cd gh-short-url
For more detail about the template, you can read the post.
Step 2: Setup the project
a. If it is the first time you are using the template please check the section A to setup necessarily things: AWS credentials, Nodejs, Python, Make, ...
b. If you have setup the template before and ensure everything is okay: Conda and its env, Python, Nodejs, Make, AWS credentials, simply run the following command:
conda activate serverless_env
npm i
Step 3: Deploy the project to make sure everything is okay
1️⃣. Activate our Conda env if you have not (just a reminder):
conda activate serverless_env
2️⃣. Edit the service name:
Open the serverless.yaml file and find the line
service: fastapi-aws-starter-kit
Replace fastapi-aws-starter-kit with anything you like, e.g: gh-shorten-url, please make sure that you do not have any existing services with the name: gh-shorten-url
3️⃣. Deploy the service:
Make sure that you have necessary permission to deploy to AWS.
Run the following command to deploy your service:
make deploy
# or if you do not have make
npx sls deploy
You will get the output like that:
Let's test our service:
curl your_service_endpoint
Perfect! Let's move on!
Step 4: Create a data file on Github
- Go to Github and create a new repo to store our shorten URLs data (choose Public)
- Important: At the time of the post, the tutorial only support public link only, please do not store your sensitive link publicly in place like Github.
- And then click on Create a new file
- We then can create a file named: data.json with its content:
{
"goo": "https://google.com"
}
Important: we need to include http or https or each url. Otherwise, it wont work.
- Click on Commit changes... to create the file.
Step 5: Develop our function
Go back to our project that we have done from Step 1 to Step 3. Please make sure that we already activated serverless_env env in case you missed it :
conda activate serverless_env
a. Add requests module:
poetry add requests
The output will be something like this:
Typically, it will update the pyproject.toml and poetry.lock file (just like package.json and package-lock.json in Nodejs).
You can use the git diff command to see the diff.
b. Update the fastapi_aws_starter_kit/fastapi_app.py file with the following content:
import requests
import json
import os
from fastapi import FastAPI, Response
from starlette.status import HTTP_404_NOT_FOUND, HTTP_301_MOVED_PERMANENTLY
from starlette.middleware.cors import CORSMiddleware
from fastapi_aws_starter_kit.config import PROJECT_NAME, API_VERSION
# Declare the application
app = FastAPI(title=PROJECT_NAME, debug=False, version=API_VERSION)
app.add_middleware(
CORSMiddleware,
allow_origins=["*"],
allow_credentials=True,
allow_methods=["*"],
allow_headers=["*"],
)
### Our main logic ###
@app.get("/shorten/{shorten_url}")
def get_original_url(shorten_url: str, response: Response):
# declare link to our data file on Github repo, you need to update to your own link
## replace sexydevops with your Github organization/user name
## replace shorten-urls-data with your reponame
data_link = 'https://raw.githubusercontent.com/sexydevops/shorten-urls-data/main/data.json'
# convert file your file data to a dict
response_json = json.loads(requests.get(data_link).text)
# find the original url based on the shorten_url argument
original_url = response_json.get(shorten_url)
# if the original url does not exist, we will return 404 status code
if original_url is None:
response.status_code = HTTP_404_NOT_FOUND
return { "message": "can not find the original url" }
# else
# - we will return status_code 301 to our function
response.status_code = HTTP_301_MOVED_PERMANENTLY
#print(original_url)
# - with a "Location" response header <= Magic begins here
# Read more about the Location response header here
# https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Location
response.headers["Location"] = original_url
return { "message": original_url }
c. Testing our function in local server:
- Open your terminal tab and if you forget to activate
serverless_envconda env, please run this:
conda activate serverless_env
- Start the offine server:
# If you have make installed on your machine:
make serve
# If you do not have make installed, run the following command instead:
npx sls offline
- Open up your browser
- Go to the link:
http://localhost:3000/dev/shorten/abcxyz, because our data file does not have the keyabcxyzsoyou will get a response like this
{
"message": "can not find the original url"
}
- Go to the link:
http://localhost:3000/dev/shorten/goo, you will be redirected to Google, because our data file has the keygoo
d. Update the data file
It is cool, right? Now you can update the content of the data file to create more shorten links.
{
"goo": "https://google.com",
"fb": "https://facebook.com",
"yt": "https://youtube.com"
}
Due to Github servers, it may take time to get the latest content of the data file.
e. Deployment:
- Do the Step 3 again!
- Grab the endpoint and you can test the app in your browsers like the section c. Testing our function in local server above
Member discussion