Ga naar inhoud

GitLab Server

System Overview

Server Details

  • Hostname: codepot.nl
  • OS: Ubuntu 20.04 LTS
  • GitLab Version: 18.3.1
  • Memory: 16GB (9.1GB in use)
  • Storage: 388GB (254GB used, 135GB available)

GitLab Services Architecture

Core Services

  • puma - Ruby on Rails application server (main application)
  • sidekiq - Background job processor for async tasks
  • gitlab-workhorse - Reverse proxy and HTTP request handler
  • nginx - Webserver and load balancer
  • postgresql - Primary database (PostgreSQL 16.8)
  • redis - Caching and session storage

Storage & Repository Management

  • gitaly - Git repository storage service
  • registry - Docker container registry (registry.codepot.nl)
  • gitlab-pages - Static site hosting (lemone.cloud)

Monitoring & Observability

  • prometheus - Metrics collection
  • alertmanager - Alert management
  • grafana - Visualization dashboard (not active)
  • gitlab-exporter - GitLab-specific metrics
  • postgres-exporter - Database metrics
  • redis-exporter - Redis metrics
  • node-exporter - System metrics

Communication & Integration

  • mailroom - Incoming email processing
  • gitlab-kas - Kubernetes Agent Server

Configuration Details

External URLs

Email Configuration

Backup Configuration

  • Local backup path: /var/opt/gitlab/backups
  • Remote backup: Google Cloud Storage (lemone-backups-codepot)
  • Retention: 3 days (259200 seconds)

Repository Storage

  • Gitaly socket: unix:/var/opt/gitlab/gitaly/gitaly.socket
  • Repository path: /var/opt/gitlab/git-data/repositories

Update Procedures

1. Pre-update Checklist

# SSH to server
ssh lemone@codepot.nl

# Check current version
sudo gitlab-rake gitlab:env:info

# Check disk space (minimum 1GB free)
df -h

# Create backup
sudo gitlab-backup create

# Check if all services are running
sudo gitlab-ctl status

2. Performing the Update

Option A: Automatic Update (APT)

# Update package lists
sudo apt update

# Check available GitLab updates
apt list --upgradable | grep gitlab

# Update GitLab
sudo apt upgrade gitlab-ce

# Reconfigure after update
sudo gitlab-ctl reconfigure

Option B: Manual Package Download

# Download specific version of GitLab package
wget https://packages.gitlab.com/gitlab/gitlab-ce/packages/ubuntu/focal/gitlab-ce_[VERSION]_amd64.deb/download.deb

# Install package
sudo dpkg -i gitlab-ce_[VERSION]_amd64.deb

# Reconfigure
sudo gitlab-ctl reconfigure

3. Post-update Verification

# Check version after update
sudo gitlab-rake gitlab:env:info

# Check if all services are running
sudo gitlab-ctl status

# Test website access
curl -I https://codepot.nl

# Check logs for errors
sudo gitlab-ctl tail

Debugging & Troubleshooting

Service Management

# Check status of all services
sudo gitlab-ctl status

# Restart specific service
sudo gitlab-ctl restart [service-name]

# Restart all services
sudo gitlab-ctl restart

# Stop all services
sudo gitlab-ctl stop

# Start all services
sudo gitlab-ctl start

Log Locations

  • All logs: /var/log/gitlab/
  • Nginx logs: /var/log/gitlab/nginx/
  • PostgreSQL logs: /var/log/gitlab/postgresql/
  • Redis logs: /var/log/gitlab/redis/
  • Puma logs: /var/log/gitlab/puma/
  • Sidekiq logs: /var/log/gitlab/sidekiq/

Live Log Monitoring

# Follow all logs
sudo gitlab-ctl tail

# Specific service logs
sudo gitlab-ctl tail nginx
sudo gitlab-ctl tail postgresql
sudo gitlab-ctl tail puma

# Log with grep filter
sudo gitlab-ctl tail | grep ERROR

Common Issues

502 Bad Gateway

# Check Puma service
sudo gitlab-ctl status puma
sudo gitlab-ctl restart puma

# Check disk space
df -h

# Check memory
free -h

Database Issues

# PostgreSQL status
sudo gitlab-ctl status postgresql

# Database console access
sudo gitlab-psql

# Database checks
sudo gitlab-rake gitlab:check

Performance Issues

# Check resource usage
htop

# GitLab performance check
sudo gitlab-rake gitlab:check

# Check configuration
sudo gitlab-ctl show-config

Configuration Management

# Backup current configuration
sudo cp /etc/gitlab/gitlab.rb /etc/gitlab/gitlab.rb.backup

# Test configuration changes
sudo gitlab-ctl reconfigure

# Rollback configuration
sudo cp /etc/gitlab/gitlab.rb.backup /etc/gitlab/gitlab.rb
sudo gitlab-ctl reconfigure

Official Documentation

Update & Maintenance

Troubleshooting

Security & Monitoring

Backup & Recovery

Notes

  • SSL certificates are managed externally via TransIP (Let's Encrypt is disabled) https://www.transip.nl/cp/domein-hosting/ssl-certificaat/prm/2087617/
  • Reverse proxy configuration via nginx['real_ip_trusted_addresses']
  • Google Cloud Storage is used for backup storage
  • Registry and Pages have separate subdomains
  • Incoming email is configured for issue creation via email

Archived Server

GitLab Archive

  • URL: https://archive.codepot.nl
  • Purpose: Read-only archive of the old GitLab host instance (cancelled April 2026)
  • Host: Hetzner CX23 + 100GB volume (Falkenstein)
  • OS: Ubuntu 24.04 LTS
  • GitLab: 18.9.5-CE (version-locked, do not upgrade)
  • Cost: ~€10/month (CX23 + volume)
  • Access: use the 1Password entry for archive.codepot.nl / old GitLab admin access
  • Secrets: /etc/gitlab/gitlab-secrets.json — see 1Password for credentials; do not store secrets in this repo
  • Config: /etc/gitlab/gitlab.rb (archive mode, no mail, no registry/pages)

What still works

Browse repositories, read issues, view MRs, search wikis.

What does not work (by design)

CI/CD pipelines, container registry, GitLab Pages, email notifications, Prometheus/Grafana monitoring.

Backups

None. This is itself the backup.

See Also