1
0
mirror of https://github.com/muety/wakapi.git synced 2023-08-10 21:12:56 +03:00
📊 A minimalist, self-hosted WakaTime-compatible backend for coding statistics
Go to file
Ferdinand Mütsch 5fc87dd143 fix: tests
2021-10-13 17:51:54 +02:00
.github feat: add test steps to Linux workflow 2021-09-17 21:24:12 +10:00
config feat: implement relay endpoint (see #237) 2021-10-11 11:00:50 +02:00
coverage fix: tests 2021-10-13 17:51:54 +02:00
data ref: embed files, bump to go 1.16 (#167) 2021-04-11 10:42:43 +00:00
middlewares fix: tests 2021-10-11 11:38:55 +02:00
migrations chore: fix syntax for postgres 2021-08-06 17:17:06 +02:00
mocks fix: tests 2021-10-13 17:51:54 +02:00
models chore: persist raw user agent value 2021-08-29 10:54:00 +02:00
postman add statusbar endpoint to postman collection 2021-09-07 13:26:26 +03:00
repositories fix: add missing non-zero field checks (fix #259) 2021-10-11 11:07:04 +02:00
routes docs: include relay endpoint in swagger docs 2021-10-13 17:47:18 +02:00
scripts fix: hotfix for invalid api base url prefix (#203) 2021-05-19 10:18:18 +02:00
services chore: invert visualization of project labels (resolve #263) 2021-10-13 17:12:55 +02:00
static docs: include relay endpoint in swagger docs 2021-10-13 17:47:18 +02:00
testing test: add api tests for query auth 2021-10-11 11:29:38 +02:00
utils fix: compatible with new wakatime-cli 2021-08-19 14:48:26 +08:00
views chore: invert visualization of project labels (resolve #263) 2021-10-13 17:12:55 +02:00
.dockerignore docker: reduce image layers 2021-01-12 21:50:20 +11:00
.gitignore feat: basic integration / api tests (wip) (resolve #9) 2021-05-28 17:14:16 +02:00
.gitpod.yml .gitpod.yml 2021-09-07 13:29:36 +00:00
config.default.yml feat: implement relay endpoint (see #237) 2021-10-11 11:00:50 +02:00
doap.rdf Add DOAP description. 2020-04-05 13:36:01 +02:00
docker-compose.yml ref: embed files, bump to go 1.16 (#167) 2021-04-11 10:42:43 +00:00
Dockerfile feat: add alpine image 2021-06-27 18:01:43 +10:00
Dockerfile.alpine fix: include tzdata package in alpine docker image [ci-skip] 2021-08-21 09:16:45 +02:00
entrypoint.sh ci: make docker container wait properly for database to come up 2021-02-06 10:43:32 +01:00
go.mod chore: notify users about failing wakatime connection 2021-08-06 23:28:03 +02:00
go.sum chore: notify users about failing wakatime connection 2021-08-06 23:28:03 +02:00
LICENSE Add license. 2019-05-22 07:32:35 +02:00
main.go feat: implement relay endpoint (see #237) 2021-10-11 11:00:50 +02:00
README.md update readme [ci skip] 2021-09-29 21:04:00 +02:00
sonar-project.properties chore: code smells 2021-06-13 10:14:15 +02:00
tailwind.config.js chore: purge tailwind classes (resolve #22) 2021-02-21 11:14:55 +01:00
version.txt chore: version 2021-10-13 17:47:33 +02:00

A minimalist, self-hosted WakaTime-compatible backend for coding statistics.

Table of Contents

Further instructions can be found in the Wiki.

📬 User Survey

I'd love to get some community feedback from active Wakapi users. If you want, please participate in the recent user survey. Thanks a lot!

🚀 Features

  • 100 % free and open-source
  • Built by developers for developers
  • Statistics for projects, languages, editors, hosts and operating systems
  • Badges
  • Weekly E-Mail Reports
  • REST API
  • Partially compatible with WakaTime
  • WakaTime integration
  • Support for Prometheus exports
  • Lightning fast
  • Self-hosted

🚧 Roadmap

Plans for the near future mainly include, besides usual improvements and bug fixes, a UI redesign as well as additional types of charts and statistics (see #101, #80, #76, #12). If you have feature requests or any kind of improvement proposals feel free to open an issue or share them in our user survey.

⌨️ How to use?

There are different options for how to use Wakapi, ranging from our hosted cloud service to self-hosting it. Regardless of which option choose, you will always have to do the client setup in addition.

☁️ Option 1: Use wakapi.dev

If you want to you out free, hosted cloud service, all you need to do is create an account and the set up your client-side tooling (see below).

However, we do not guarantee data persistence, so you might potentially lose your data if the service is taken down some day

📦 Option 2: Quick-run a Release

$ curl -L https://wakapi.dev/get | bash

🐳 Option 3: Use Docker

# Create a persistent volume
$ docker volume create wakapi-data

# Run the container
$ docker run -d \
  -p 3000:3000 \
  -e "WAKAPI_PASSWORD_SALT=$(cat /dev/urandom | tr -dc 'a-zA-Z0-9' | fold -w ${1:-32} | head -n 1)" \
  -v wakapi-data:/data \
  --name wakapi n1try/wakapi

Note: By default, SQLite is used as a database. To run Wakapi in Docker with MySQL or Postgres, see Dockerfile and config.default.yml for further options.

If you want to run Wakapi on Kubernetes, there is wakapi-helm-chart for quick and easy deployment.

🧑‍💻 Option 4: Compile and run from source

Prerequisites

  • Go >= 1.16 (with $GOPATH properly set)
  • gcc (to compile go-sqlite3)
    • Fedora / RHEL: dnf install @development-tools
    • Ubuntu / Debian: apt install build-essential
    • Windows: See here

Compile & Run

# Build the executable
$ go build -o wakapi

# Adapt config to your needs
$ cp config.default.yml config.yml
$ vi config.yml

# Run it
$ ./wakapi

Note: Check the comments config.yml for best practices regarding security configuration and more.

💻 Client Setup

Wakapi relies on the open-source WakaTime client tools. In order to collect statistics to Wakapi, you need to set them up.

  1. Set up WakaTime for your specific IDE or editor. Please refer to the respective plugin guide
  2. Editing your local ~/.wakatime.cfg file as follows
[settings]

# Your Wakapi server URL or 'https://wakapi.dev' when using the cloud server
api_url = http://localhost:3000/api/heartbeat

# Your Wakapi API key (get it from the web interface after having created an account)
api_key = 406fe41f-6d69-4183-a4cc-121e0c524c2b

Optionally, you can set up a client-side proxy in addition.

🔧 Configuration Options

You can specify configuration options either via a config file (default: config.yml, customziable through the -c argument) or via environment variables. Here is an overview of all options.

YAML Key Environment Variable Default Description
env ENVIRONMENT dev Whether to use development- or production settings
app.custom_languages - - Map from file endings to language names
server.port WAKAPI_PORT 3000 Port to listen on
server.listen_ipv4 WAKAPI_LISTEN_IPV4 127.0.0.1 IPv4 network address to listen on (leave blank to disable IPv4)
server.listen_ipv6 WAKAPI_LISTEN_IPV6 ::1 IPv6 network address to listen on (leave blank to disable IPv6)
server.listen_socket WAKAPI_LISTEN_SOCKET - UNIX socket to listen on (leave blank to disable UNIX socket)
server.timeout_sec WAKAPI_TIMEOUT_SEC 30 Request timeout in seconds
server.tls_cert_path WAKAPI_TLS_CERT_PATH - Path of SSL server certificate (leave blank to not use HTTPS)
server.tls_key_path WAKAPI_TLS_KEY_PATH - Path of SSL server private key (leave blank to not use HTTPS)
server.base_path WAKAPI_BASE_PATH / Web base path (change when running behind a proxy under a sub-path)
security.password_salt WAKAPI_PASSWORD_SALT - Pepper to use for password hashing
security.insecure_cookies WAKAPI_INSECURE_COOKIES false Whether or not to allow cookies over HTTP
security.cookie_max_age WAKAPI_COOKIE_MAX_AGE 172800 Lifetime of authentication cookies in seconds or 0 to use Session cookies
security.allow_signup WAKAPI_ALLOW_SIGNUP true Whether to enable user registration
security.expose_metrics WAKAPI_EXPOSE_METRICS false Whether to expose Prometheus metrics under /api/metrics
db.host WAKAPI_DB_HOST - Database host
db.port WAKAPI_DB_PORT - Database port
db.user WAKAPI_DB_USER - Database user
db.password WAKAPI_DB_PASSWORD - Database password
db.name WAKAPI_DB_NAME wakapi_db.db Database name
db.dialect WAKAPI_DB_TYPE sqlite3 Database type (one of sqlite3, mysql, postgres, cockroach)
db.charset WAKAPI_DB_CHARSET utf8mb4 Database connection charset (for MySQL only)
db.max_conn WAKAPI_DB_MAX_CONNECTIONS 2 Maximum number of database connections
db.ssl WAKAPI_DB_SSL false Whether to use TLS encryption for database connection (Postgres and CockroachDB only)
db.automgirate_fail_silently WAKAPI_DB_AUTOMIGRATE_FAIL_SILENTLY false Whether to ignore schema auto-migration failures when starting up
mail.enabled WAKAPI_MAIL_ENABLED true Whether to allow Wakapi to send e-mail (e.g. for password resets)
mail.sender WAKAPI_MAIL_SENDER noreply@wakapi.dev Default sender address for outgoing mails (ignored for MailWhale)
mail.provider WAKAPI_MAIL_PROVIDER smtp Implementation to use for sending mails (one of [smtp, mailwhale])
mail.smtp.* WAKAPI_MAIL_SMTP_* - Various options to configure SMTP. See default config for details
mail.mailwhale.* WAKAPI_MAIL_MAILWHALE_* - Various options to configure MailWhale sending service. See default config for details
sentry.dsn WAKAPI_SENTRY_DSN DSN for to integrate Sentry for error logging and tracing (leave empty to disable)
sentry.enable_tracing WAKAPI_SENTRY_TRACING false Whether to enable Sentry request tracing
sentry.sample_rate WAKAPI_SENTRY_SAMPLE_RATE 0.75 Probability of tracing a request in Sentry
sentry.sample_rate_heartbeats WAKAPI_SENTRY_SAMPLE_RATE_HEARTBEATS 0.1 Probability of tracing a heartbeats request in Sentry

Supported databases

Wakapi uses GORM as an ORM. As a consequence, a set of different relational databases is supported.

  • SQLite (default, easy setup)
  • MySQL (recommended, because most extensively tested)
  • MariaDB (open-source MySQL alternative)
  • Postgres (open-source as well)
  • CockroachDB (cloud-native, distributed, Postgres-compatible API)

Client-side proxy (optional)

See the advanced setup instructions.

🔧 API Endpoints

See our Swagger API Documentation.

Generating Swagger docs

$ go get -u github.com/swaggo/swag/cmd/swag
$ swag init -o static/docs

🤝 Integrations

Prometheus Export

You can export your Wakapi statistics to Prometheus to view them in a Grafana dashboard or so. Here is how.

# 1. Start Wakapi with the feature enabled
$ export WAKAPI_EXPOSE_METRICS=true
$ ./wakapi

# 2. Get your API key and hash it
$ echo "<YOUR_API_KEY>" | base64

# 3. Add a Prometheus scrape config to your prometheus.yml (see below)

Scrape config example

# prometheus.yml
# (assuming your Wakapi instance listens at localhost, port 3000)

scrape_configs:
  - job_name: 'wakapi'
    scrape_interval: 1m
    metrics_path: '/api/metrics'
    bearer_token: '<YOUR_BASE64_HASHED_TOKEN>'
    static_configs:
      - targets: ['localhost:3000']

Grafana

There is also a nice Grafana dashboard, provided by the author of wakatime_exporter.

WakaTime Integration

Wakapi plays well together with WakaTime. For one thing, you can forward heartbeats from Wakapi to WakaTime to effectively use both services simultaneously. In addition, there is the option to import historic data from WakaTime for consistency between both services. Both features can be enabled in the Integrations section of your Wakapi instance's settings page.

GitHub Readme Stats Integrations

Wakapi also integrates with GitHub Readme Stats to generate fancy cards for you. Here is an example.

Click to view code
![](https://github-readme-stats.vercel.app/api/wakatime?username={yourusername}&api_domain=wakapi.dev&bg_color=2D3748&title_color=2F855A&icon_color=2F855A&text_color=ffffff&custom_title=Wakapi%20Week%20Stats&layout=compact)

Github Readme Metrics Integration

There is a WakaTime plugin for GitHub metrics that is also compatible with Wakapi.

Preview:

Click to view code
- uses: lowlighter/metrics@latest
  with:
    # ... other options
    plugin_wakatime: yes
    plugin_wakatime_token: ${{ secrets.WAKATIME_TOKEN }}      # Required
    plugin_wakatime_days: 7                                   # Display last week stats
    plugin_wakatime_sections: time, projects, projects-graphs # Display time and projects sections, along with projects graphs
    plugin_wakatime_limit: 4                                  # Show 4 entries per graph
    plugin_wakatime_url: http://wakapi.dev                  # Wakatime url endpoint
    plugin_wakatime_user: .user.login                         # User


👍 Best Practices

It is recommended to use wakapi behind a reverse proxy, like Caddy or nginx to enable TLS encryption (HTTPS). However, if you want to expose your wakapi instance to the public anyway, you need to set server.listen_ipv4 to 0.0.0.0 in config.yml

🧪 Tests

Unit Tests

Unit tests are supposed to test business logic on a fine-grained level. They are implemented as part of the application, using Go's testing package alongside stretchr/testify.

How to run

$ CGO_FLAGS="-g -O2 -Wno-return-local-addr" go test -json -coverprofile=coverage/coverage.out ./... -run ./...

API Tests

API tests are implemented as black box tests, which interact with a fully-fledged, standalone Wakapi through HTTP requests. They are supposed to check Wakapi's web stack and endpoints, including response codes, headers and data on a syntactical level, rather than checking the actual content that is returned.

Our API (or end-to-end, in some way) tests are implemented as a Postman collection and can be run either from inside Postman, or using newman as a command-line runner.

To get a predictable environment, tests are run against a fresh and clean Wakapi instance with a SQLite database that is populated with nothing but some seed data (see data.sql). It is usually recommended for software tests to be safe, stateless and without side effects. In contrary to that paradigm, our API tests strictly require a fixed execution order (which Postman assures) and their assertions may rely on specific previous tests having succeeded.

Prerequisites (Linux only)

# 1. sqlite (cli)
$ sudo apt install sqlite  # Fedora: sudo dnf install sqlite

# 2. newman
$ npm install -g newman

How to run (Linux only)

$ ./testing/run_api_tests.sh

🤓 Developer Notes

Building web assets

To keep things minimal, Wakapi does not contain a package.json, node_modules or any sort of frontend build step. Instead, all JS and CSS assets are included as static files and checked in to Git. This way we can avoid requiring NodeJS to build Wakapi. However, for TailwindCSS it makes sense to run it through a "build" step to benefit from purging and significantly reduce it in size. To only require this at the time of development, the compiled asset is checked in to Git as well. Similarly, Iconify bundles are also created at development time and checked in to the repo.

TailwindCSS

$ tailwindcss-cli build static/assets/vendor/tailwind.css -o static/assets/vendor/tailwind.dist.css

Iconify

$ yarn add -D @iconify/json-tools @iconify/json
$ node scripts/bundle_icons.js

New icons can be added by editing the icons array in scripts/bundle_icons.js.

FAQs

Since Wakapi heavily relies on the concepts provided by WakaTime, their FAQs apply to Wakapi for large parts as well. You might find answers there.

What data is sent to Wakapi?
  • File names
  • Project names
  • Editor names
  • You computer's host name
  • Timestamps for every action you take in your editor
  • ...

See the related WakaTime FAQ section for details.

If you host Wakapi yourself, you have control over all your data. However, if you use our webservice and are concerned about privacy, you can also exclude or obfuscate certain file- or project names.

What happens if I'm offline?

All data is cached locally on your machine and sent in batches once you're online again.

How did Wakapi come about?

Wakapi was started when I was a student, who wanted to track detailed statistics about my coding time. Although I'm a big fan of WakaTime I didn't want to pay 9 $ a month back then. Luckily, most parts of WakaTime are open source!

How does Wakapi compare to WakaTime?

Wakapi is a small subset of WakaTime and has a lot less features. Cool WakaTime features, that are missing Wakapi, include:

  • Leaderboards
  • Embeddable Charts
  • Personal Goals
  • Team- / Organization Support
  • Integrations (with GitLab, etc.)
  • Richer API

WakaTime is worth the price. However, if you only want basic statistics and keep sovereignty over your data, you might want to go with Wakapi.

How are durations calculated?

Inferring a measure for your coding time from heartbeats works a bit different than in WakaTime. While WakaTime has timeout intervals, Wakapi essentially just pads every heartbeat, that occurs after a longer pause, with 2 extra minutes.

Here is an example (circles are heartbeats):

|---o---o--------------o---o---|
|   |10s|      3m      |10s|   |

It is unclear how to handle the three minutes in between. Did the developer do a 3-minute break or were just no heartbeats being sent, e.g. because the developer was starring at the screen find a solution, but not actually typing code.

  • WakaTime (with 5 min timeout): 3 min 20 sec
  • WakaTime (with 2 min timeout): 20 sec
  • Wakapi: 10 sec + 2 min + 10 sec = 2 min 20 sec

Wakapi adds a "padding" of two minutes before the third heartbeat. This is why total times will slightly vary between Wakapi and WakaTime.

🌳 Treeware

This package is Treeware. If you use it in production, then we ask that you buy the world a tree to thank us for our work. By contributing to the Treeware forest youll be creating employment for local families and restoring wildlife habitats.

👏 Support

Coding in open source is my passion and I would love to do it on a full-time basis and make a living from it one day. So if you like this project, please consider supporting it 🙂. You can donate either through buying me a coffee or becoming a GitHub sponsor. Every little donation is highly appreciated and boosts my motivation to keep improving Wakapi!

🙏 Thanks

I highly appreciate the efforts of @alanhamlett and the WakaTime team and am thankful for their software being open source.

Moreover, thanks to JetBrains for supporting this project as part of their open-source program.

📓 License

GPL-v3 @ Ferdinand Mütsch