mirror/liteshort
1
0
Fork 0
mirror of https://git.ikl.sh/132ikl/liteshort.git synced 2023-08-10 21:13:04 +03:00
Code Issues Projects Releases Wiki Activity

Release v1.2.0 (#21)

Browse Source 
Release v1.2.0
This commit is contained in:
132ikl 2020-04-10 05:38:59 -04:00 committed by GitHub
commit 968d0e77ac
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
16 changed files with 265 additions and 304 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

124
README.md
View File

@ -7,129 +7,11 @@ Click [here](https://ls.ikl.sh) for a live demo.
liteshort is designed with the main goal of being lightweight. It does away with all the frills of other link shorteners and allows the best of the basics at a small resource price. liteshort uses under 20 MB of memory idle, per worker. liteshort has an easy-to-use API and web interface. liteshort doesn't store any more information than necessary: just the long and short URLs. It does not log the date of creation, the remote IP, or any other information.
liteshort uses Python 3, [Flask](http://flask.pocoo.org/), SQLite3, and [uwsgi](https://uwsgi-docs.readthedocs.io/en/latest/) for the backend.
The frontend is a basic POST form using [PureCSS](https://purecss.io).
liteshort focuses on being configurable. There are over 15 config options and most updates will add more. Configuration is done through the easy-to-use YAML format.
![liteshort screenshot](https://fs.ikl.sh/selif/4cgndb6e.png)
## Installation
This installation procedure assumes that you plan to installing using a web server reverse proxy through a unix socket. This guide is loosely based upon DigitalOcean's [Flask/uWSGI/nginx guide](https://www.digitalocean.com/community/tutorials/how-to-serve-flask-applications-with-uswgi-and-nginx-on-ubuntu-18-04).
Before installing, you must the following or your distribution's equivalent must be installed:
* python3-pip
* python3-dev
* python3-setuptools
* python3-venv
* build-essential
Start in the directory you wish to install to and modify to fit your installation. It is recommended to use a user specifically for liteshort and the www-data group for your installation folder.
```sh
git clone https://github.com/132ikl/liteshort
python3 -m venv virtualenv
source virtualenv/bin/activate
pip install wheel
pip install bcrypt flask pyyaml uwsgi
```
Edit `liteshort.ini` and `liteshort.service` as seen fit. Then edit `config.yml` according to the [Configuration](#configuration) section.
Finally,
```sh
cp liteshort.service /etc/systemd/system/
systemctl enable liteshort
systemctl start liteshort
```
liteshort is now accessible through a reverse proxy. The socket file is created in the install path.
## Configuration
The configuration file has an explanation for each option. This section will detail the mandatory options to be set before the program is able to be started.
`admin_hashed_password` or `admin_password`
* These must be set in order to use the API. If you do not care about the API, simply set `disable_api` to true.
As to not store the API password in cleartext, `admin_hashed_password` is preferred over `admin_password`. Run `securepass.sh` in order to generate the password hash. Set `admin_hashed_password` to the output of the script, excluding the username header at the beginning of the hash.
Note that using admin_hashed_password is more resource-intensive than `admin_password`, so the API will be noticeably slower when using `admin_hashed_password`.
`secret_key`
* This is used for cookies in order to store messages between requests. It should be a randomized key 12-16 characters, comprised of letters, number, and symbols. A standard password from a generator works fine.
## API
All API requests should have the POST form data `format` set to `json`.
In order to create a new short URL, simply make a POST request with the form data `long` set to your long link and, optionally, set `short` to your short link.
Everything other than creation of links requires BasicAuth using the username and password defined in the configuration file. To use the following commands, set `api` to the command in the form data of your request.
* `list` and `listshort`
* Lists all links the the database, sorted by short links.
* `listlong`
* Lists all links in the database, sorted by long links.
* `delete`
* Deletes a URL. In the form data, set `short` to the short link you want to delete, or set `long` to delete all short links that redirect to the provided long link.
### Example Request
```
curl -u [admin_username]:[admin_password] \
-d 'format=json' \
-d 'api=delete' \
-d 'short=[short]' \
[url]
```
## Using a reverse proxy
The following are barebones examples of an nginx proxy for liteshort, meaning it doesn't have SSL or anything fancy. You may also use a non-nginx webserver by making a config equivalent for it based upon the following configurations. Make sure your webserver is serving the /static/ folder. While liteshort can serve the folder, webservers are much more efficient at serving static files.
### On domain root
```
server {
listen 80;
server_name example.com;
location ^~ /static/ {
include /etc/nginx/mime.types;
root /usr/local/liteshort/;
}
location / {
include uwsgi_params;
uwsgi_pass unix:/path/to/liteshort/liteshort.sock;
}
}
```
### On a subdomain
First, make sure `site_domain` and `subdomain` options are set in config.yml. If you want the web interface on a subdomain, but the actual shortlinks on the main domain, as seen on the [demo site](https://ls.ikl.sh), use a configuration akin to the following. Make sure that anything you want to happen before liteshort, like a homepage on /, has its location block BEFORE the rewrite block. Nginx goes in numerical order of location blocks, so the rewrite location block will redirect everything on / to liteshort if not the last block.
```
server {
listen 80;
server_name subdomain.example.com;
location / {
include uwsgi_params;
uwsgi_pass unix:/path/to/liteshort/liteshort.sock;
}
}
server {
listen 80;
server_name example.com;
location ^~ /static/ {
include /etc/nginx/mime.types;
root /usr/local/liteshort/;
}
location / {
rewrite /example/subdomain.example(.+) /$1 break;
include uwsgi_params;
uwsgi_pass unix:/usr/local/liteshort/liteshort.sock;
}
}
```
Liteshort's installation process is dead-simple. Check out the [installation page](https://github.com/132ikl/liteshort/wiki/How-to-Install) for info on how to install.

13
liteshort.service
View File

@ -1,13 +0,0 @@
[Unit]
Description=uWSGI instance to serve liteshort
After=network.target
[Service]
User=
Group=www-data
WorkingDirectory=/path/to/install
Environment="PATH=/path/to/install/virtualenv/bin"
ExecStart=/path/to/install/virtualenv/bin/uwsgi --ini liteshort.ini
[Install]
WantedBy=multi-user.target

0
liteshort/__init__.py Normal file
View File

115
liteshort/config.py Normal file
View File

@ -0,0 +1,115 @@
import logging
from pathlib import Path
from shutil import copyfile
from appdirs import site_config_dir, user_config_dir
from pkg_resources import resource_filename
from yaml import safe_load
logging.basicConfig(level=logging.DEBUG)
LOGGER = logging.getLogger(__name__)
def get_config():
APP = "liteshort"
AUTHOR = "132ikl"
paths = [
Path("/etc/liteshort"),
Path(site_config_dir(APP, AUTHOR)),
Path(user_config_dir(APP, AUTHOR)),
Path(),
]
for path in paths:
f = path / "config.yml"
if f.exists():
LOGGER.debug(f"Selecting config file {f}")
return open(f, "r")
for path in paths:
try:
path.mkdir(exist_ok=True)
template = resource_filename(__name__, "config.template.yml")
copyfile(template, (path / "config.template.yml"))
copyfile(template, (path / "config.yml"))
return open(path / "config.yml", "r")
except (PermissionError, OSError) as e:
LOGGER.warn(f"Failed to create config in {path}")
LOGGER.debug("", exc_info=True)
raise FileNotFoundError("Cannot find config.yml, and failed to create it")
# TODO: yikes
def load_config():
with get_config() as config:
configYaml = safe_load(config)
config = {
k.lower(): v for k, v in configYaml.items()
} # Make config keys case insensitive
req_options = {
"admin_username": "admin",
"database_name": "urls",
"random_length": 4,
"allowed_chars": "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_",
"random_gen_timeout": 5,
"site_name": "liteshort",
"site_domain": None,
"show_github_link": True,
"secret_key": None,
"disable_api": False,
"subdomain": "",
"latest": "l",
"selflinks": False,
"blocklist": [],
}
config_types = {
"admin_username": str,
"database_name": str,
"random_length": int,
"allowed_chars": str,
"random_gen_timeout": int,
"site_name": str,
"site_domain": (str, type(None)),
"show_github_link": bool,
"secret_key": str,
"disable_api": bool,
"subdomain": (str, type(None)),
"latest": (str, type(None)),
"selflinks": bool,
"blocklist": list,
}
for option in req_options.keys():
if (
option not in config.keys()
): # Make sure everything in req_options is set in config
config[option] = req_options[option]
for option in config.keys():
if option in config_types:
matches = False
if type(config_types[option]) is not tuple:
config_types[option] = (
config_types[option],
) # Automatically creates tuple for non-tuple types
for req_type in config_types[
option
]: # Iterates through tuple to allow multiple types for config options
if type(config[option]) is req_type:
matches = True
if not matches:
raise TypeError(option + " is incorrect type")
if not config["disable_api"]:
if "admin_hashed_password" in config.keys() and config["admin_hashed_password"]:
config["password_hashed"] = True
elif "admin_password" in config.keys() and config["admin_password"]:
config["password_hashed"] = False
else:
raise TypeError(
"admin_password or admin_hashed_password must be set in config.yml"
)
return config

17
config.yml → liteshort/config.template.yml
View File

@ -5,13 +5,13 @@ admin_username: 'admin'
# String: Plaintext password to make admin API requests
# Safe to remove if admin_hashed_password is set
# Default: unset
#admin_password:
admin_password: CHANGE_ME
# String: Hashed password (bcrypt) to make admin API requests - Preferred over plaintext, use securepass.sh to generate
# String: Hashed password (bcrypt) to make admin API requests - Preferred over plaintext, use lshash to generate
# Please note that authentication takes noticeably longer than using plaintext password
# Don't include the <username>: segment, just the hash
# Default: unset (required to start application)
admin_hashed_password:
#admin_hashed_password:
# Boolean: Disables API. If set to true, admin_password/admin_hashed_password do not need to be set.
# Default: false
@ -20,7 +20,7 @@ disable_api: false
# String: Secret key used for cookies (used for storage of messages)
# This should be a 12-16 character randomized string with letters, numbers, and symbols
# Default: unset (required to start application)
secret_key:
secret_key: CHANGE_ME
# String: Filename of the URL database without extension
# Default: 'urls'
@ -56,6 +56,7 @@ subdomain:
# String: URL which takes you to the most recent short URL's destination
# Short URLs cannot be created with this string if set
# Unset to disable
# Default: l
latest: 'l'
@ -66,3 +67,11 @@ show_github_link: true
# Boolean: Allow short URLs linking to your site_domain URL
# Default: false
selflinks: false
# List: Prevent creation of URLs linking to domains in the blocklist
# Example of list formatting in yaml:
# blocklist:
# - blocklisted.com
# - subdomain.blocklisted.net
# Default: []
blocklist: []

209
liteshort.py → liteshort/main.py
View File

@ -1,94 +1,16 @@
# Copyright (c) 2019 Steven Spangler <132@ikl.sh>
# This file is part of liteshort by 132ikl
# This software is license under the MIT license. It should be included in your copy of this software.
# A copy of the MIT license can be obtained at https://mit-license.org/
import os
import random
import sqlite3
import time
import urllib
import bcrypt
import yaml
from flask import (Flask, current_app, flash, g, jsonify, make_response,
redirect, render_template, request, send_from_directory,
url_for)
import flask
from bcrypt import checkpw
from flask import current_app, g, redirect, render_template, request, url_for
app = Flask(__name__)
from .config import load_config
def load_config():
new_config = yaml.load(open("config.yml"))
new_config = {
k.lower(): v for k, v in new_config.items()
} # Make config keys case insensitive
req_options = {
"admin_username": "admin",
"database_name": "urls",
"random_length": 4,
"allowed_chars": "ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789-_",
"random_gen_timeout": 5,
"site_name": "liteshort",
"site_domain": None,
"show_github_link": True,
"secret_key": None,
"disable_api": False,
"subdomain": "",
"latest": "l",
"selflinks": False,
}
config_types = {
"admin_username": str,
"database_name": str,
"random_length": int,
"allowed_chars": str,
"random_gen_timeout": int,
"site_name": str,
"site_domain": (str, type(None)),
"show_github_link": bool,
"secret_key": str,
"disable_api": bool,
"subdomain": (str, type(None)),
"latest": (str, type(None)),
"selflinks": bool,
}
for option in req_options.keys():
if (
option not in new_config.keys()
): # Make sure everything in req_options is set in config
new_config[option] = req_options[option]
for option in new_config.keys():
if option in config_types:
matches = False
if type(config_types[option]) is not tuple:
config_types[option] = (
config_types[option],
) # Automatically creates tuple for non-tuple types
for req_type in config_types[
option
]: # Iterates through tuple to allow multiple types for config options
if type(new_config[option]) is req_type:
matches = True
if not matches:
raise TypeError(option + " is incorrect type")
if not new_config["disable_api"]:
if (
"admin_hashed_password" in new_config.keys()
and new_config["admin_hashed_password"]
):
new_config["password_hashed"] = True
elif "admin_password" in new_config.keys() and new_config["admin_password"]:
new_config["password_hashed"] = False
else:
raise TypeError(
"admin_password or admin_hashed_password must be set in config.yml"
)
return new_config
app = flask.Flask(__name__)
def authenticate(username, password):
@ -115,15 +37,19 @@ def check_short_exist(short): # Allow to also check against a long link
return False
def check_self_link(long):
if get_baseUrl().rstrip("/") in long:
def linking_to_blocklist(long):
# Removes protocol and other parts of the URL to extract the domain name
long = long.split("//")[-1].split("/")[0]
if long in current_app.config["blocklist"]:
return True
if not current_app.config["selflinks"]:
return long in get_baseUrl()
return False
def check_password(password, pass_config):
if pass_config["password_hashed"]:
return bcrypt.checkpw(
return checkpw(
password.encode("utf-8"),
pass_config["admin_hashed_password"].encode("utf-8"),
)
@ -133,7 +59,7 @@ def check_password(password, pass_config):
raise RuntimeError("This should never occur! Bailing...")
def delete_url(deletion):
def delete_short(deletion):
result = query_db(
"SELECT * FROM urls WHERE short = ?", (deletion,), False, None
) # Return as tuple instead of row
@ -142,6 +68,18 @@ def delete_url(deletion):
return len(result)
def delete_long(long):
if "//" in long:
long = long.split("//")[-1]
long = "%" + long + "%"
result = query_db(
"SELECT * FROM urls WHERE long LIKE ?", (long,), False, None
) # Return as tuple instead of row
get_db().cursor().execute("DELETE FROM urls WHERE long LIKE ?", (long,))
get_db().commit()
return len(result)
def dict_factory(cursor, row):
d = {}
for idx, col in enumerate(cursor.description):
@ -191,23 +129,18 @@ def nested_list_to_dict(l):
def response(rq, result, error_msg="Error: Unknown error"):
if rq.form.get("api") and not rq.form.get("format") == "json":
return "Format type HTML (default) not support for API" # Future-proof for non-json return types
if rq.form.get("format") == "json":
# If not result provided OR result doesn't exist, send error
# Allows for setting an error message with explicitly checking in regular code
if result:
if result is True: # Allows sending with no result (ie. during deletion)
return jsonify(success=True)
else:
return jsonify(success=True, result=result)
if rq.form.get("api"):
if rq.accept_mimetypes.accept_json:
if result:
return flask.jsonify(success=bool(result), result=result)
return flask.jsonify(success=bool(result), message=error_msg)
else:
return jsonify(success=False, error=error_msg)
return "Format type HTML (default) not supported for API" # Future-proof for non-json return types
else:
if result and result is not True:
flash(result, "success")
flask.flash(result, "success")
elif not result:
flash(error_msg, "error")
flask.flash(error_msg, "error")
return render_template("main.html")
@ -281,7 +214,7 @@ app.config["SERVER_NAME"] = app.config["site_domain"]
@app.route("/favicon.ico", subdomain=app.config["subdomain"])
def favicon():
return send_from_directory(
return flask.send_from_directory(
os.path.join(app.root_path, "static"),
"favicon.ico",
mimetype="image/vnd.microsoft.icon",
@ -297,16 +230,49 @@ def main():
def main_redir(url):
long = get_long(url)
if long:
resp = make_response(redirect(long, 301))
resp = flask.make_response(flask.redirect(long, 301))
else:
flash('Short URL "' + url + "\" doesn't exist", "error")
resp = make_response(redirect(url_for("main")))
flask.flash('Short URL "' + url + "\" doesn't exist", "error")
resp = flask.make_response(flask.redirect(url_for("main")))
resp.headers.set("Cache-Control", "no-store, must-revalidate")
return resp
@app.route("/", methods=["POST"], subdomain=app.config["subdomain"])
def main_post():
if request.form.get("api"):
if current_app.config["disable_api"]:
return response(request, None, "API is disabled.")
# All API calls require authentication
if not request.authorization or not authenticate(
request.authorization["username"], request.authorization["password"]
):
return response(request, None, "BaiscAuth failed")
command = request.form["api"]
if command == "list" or command == "listshort":
return response(request, list_shortlinks(), "Failed to list items")
elif command == "listlong":
shortlinks = list_shortlinks()
shortlinks = {v: k for k, v in shortlinks.items()}
return response(request, shortlinks, "Failed to list items")
elif command == "delete":
deleted = 0
if "long" not in request.form and "short" not in request.form:
return response(request, None, "Provide short or long in POST data")
if "short" in request.form:
deleted = delete_short(request.form["short"]) + deleted
if "long" in request.form:
deleted = delete_long(request.form["long"]) + deleted
if deleted > 0:
return response(
request,
"Deleted " + str(deleted) + " URL" + ("s" if deleted > 1 else ""),
)
else:
return response(request, None, "URL not found")
else:
return response(request, None, "Command " + command + " not found")
if request.form.get("long"):
if not validate_long(request.form["long"]):
return response(request, None, "Long URL is not valid")
@ -328,10 +294,7 @@ def main_post():
if check_short_exist(short):
return response(request, None, "Short URL already taken")
long_exists = check_long_exist(request.form["long"])
if (
check_self_link(request.form["long"])
and not current_app.config["selflinks"]
):
if linking_to_blocklist(request.form["long"]):
return response(request, None, "You cannot link to this site")
if long_exists and not request.form.get("short"):
set_latest(request.form["long"])
@ -346,37 +309,7 @@ def main_post():
)
set_latest(request.form["long"])
get_db().commit()
return response(request, get_baseUrl() + short, "Error: Failed to generate")
elif request.form.get("api"):
if current_app.config["disable_api"]:
return response(request, None, "API is disabled.")
# All API calls require authentication
if not request.authorization or not authenticate(
request.authorization["username"], request.authorization["password"]
):
return response(request, None, "BaiscAuth failed")
command = request.form["api"]
if command == "list" or command == "listshort":
return response(request, list_shortlinks(), "Failed to list items")
elif command == "listlong":
shortlinks = list_shortlinks()
shortlinks = {v: k for k, v in shortlinks.items()}
return response(request, shortlinks, "Failed to list items")
elif command == "delete":
deleted = 0
if "long" not in request.form and "short" not in request.form:
return response(request, None, "Provide short or long in POST data")
if "short" in request.form:
deleted = delete_url(request.form["short"]) + deleted
if "long" in request.form:
deleted = delete_url(request.form["long"]) + deleted
if deleted > 0:
return response(request, "Deleted " + str(deleted) + " URLs")
else:
return response(request, None, "Failed to delete URL")
else:
return response(request, None, "Command " + command + " not found")
else:
return response(request, None, "Long URL required")

0
static/GitHub.svg → liteshort/static/GitHub.svg
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 1.4 KiB

After

Width:  |  Height:  |  Size: 1.4 KiB

0
static/favicon.ico → liteshort/static/favicon.ico
View File

Side by Side Swipe Overlay

Before

Width:  |  Height:  |  Size: 15 KiB

After

Width:  |  Height:  |  Size: 15 KiB

0
static/styles.css → liteshort/static/styles.css
View File

0
templates/main.html → liteshort/templates/main.html
View File

20
liteshort/util.py Normal file
View File

@ -0,0 +1,20 @@
from getpass import getpass
import bcrypt
def hash_passwd():
salt = bcrypt.gensalt()
try:
unhashed = getpass("Type password to hash: ")
unhashed2 = getpass("Confirm: ")
except (KeyboardInterrupt, EOFError):
pass
if unhashed != unhashed2:
print("Passwords don't match.")
return None
hashed = bcrypt.hashpw(unhashed.encode("utf-8"), salt)
print("Password hash: " + hashed.decode("utf-8"))

2
wsgi.py → liteshort/wsgi.py
View File

@ -1,4 +1,4 @@
from liteshort import app
from .main import app
if __name__ == "__main__":
app.run()

24
securepass.sh
View File

@ -1,24 +0,0 @@
#!/bin/sh
## bcrypt passwd generator ##
#############################
CMD=$(which htpasswd 2>/dev/null)
OPTS="-nBC 12"
read -p "Username: " USERNAME
check_config() {
if [ -z $CMD ]; then
printf "Exiting: htpasswd is missing.\n"
exit 1
fi
if [ -z "$USERNAME" ]; then
usage
fi
}
check_config $USERNAME
printf "Generating Bcrypt hash for username: $USERNAME\n\n"
$CMD $OPTS $USERNAME
exit $?

30
setup.py Normal file
View File

@ -0,0 +1,30 @@
import setuptools
with open("README.md", "r") as fh:
long_description = fh.read()
setuptools.setup(
name="liteshort",
version="1.2.0",
author="Steven Spangler",
author_email="132@ikl.sh",
description="User-friendly, actually lightweight, and configurable URL shortener",
long_description=long_description,
long_description_content_type="text/markdown",
url="https://github.com/132ikl/liteshort",
packages=setuptools.find_packages(),
package_data={"liteshort": ["templates/*", "static/*", "config.template.yml"]},
entry_points={
"console_scripts": [
"liteshort = liteshort.main:app.run",
"lshash = liteshort.util:hash_passwd",
]
},
classifiers=[
"Programming Language :: Python :: 3",
"License :: OSI Approved :: MIT License",
"Operating System :: POSIX :: Linux",
],
install_requires=["flask~=1.1.2", "bcrypt~=3.1.7", "pyyaml", "appdirs~=1.4.3"],
python_requires=">=3.7",
)

6
liteshort.ini → setup/liteshort.ini
View File

@ -1,12 +1,12 @@
[uwsgi]
module = wsgi:app
module = liteshort.wsgi:app
plugin = python3
master = true
processes = 2
socket = liteshort.sock
socket = /run/liteshort.sock
chmod-socket = 666
vacuum = true
die-on-term = true

9
setup/liteshort.service Normal file
View File

@ -0,0 +1,9 @@
[Unit]
Description=uWSGI instance to serve liteshort
After=network.target
[Service]
ExecStart=/usr/bin/uwsgi --ini /etc/liteshort/liteshort.ini
[Install]
WantedBy=multi-user.target