Data

Tools

Indexes

Applications

Experiments

Repos

An open API service providing repository metadata for many open source software ecosystems.

GitHub / postgres-ai / database-lab-engine

DBLab enables 🖖 database branching and ⚡️ thin cloning for any Postgres database and empowers DB testing in CI/CD. This optimizes database-related costs while improving time-to-market and software quality. Follow to stay updated.

JSON API: http://repos.ecosyste.ms/api/v1/hosts/GitHub/repositories/postgres-ai%2Fdatabase-lab-engine
PURL: pkg:github/postgres-ai/database-lab-engine

Stars: 2,213
Forks: 61
Open issues: 16

License: apache-2.0
Language: Go
Size: 21.4 MB
Dependencies parsed at: Pending

Created at: over 5 years ago
Updated at: about 1 month ago
Pushed at: about 1 month ago
Last synced at: about 1 month ago

Commit Stats

Commits: 794
Authors: 29
Mean commits per author: 27.38
Development Distribution Score: 0.471
More commit stats: https://commits.ecosyste.ms/hosts/GitHub/repositories/postgres-ai/database-lab-engine

Topics: branching, ci, ci-cd, cicd, cloning, database, database-branching, database-migration, database-migrations, database-schema, database-schema-evolutions, database-schema-migration, pgsql, postgres, postgresql, sql

Funding Links https://github.com/sponsors/postgres-ai, https://patreon.com/Database_Lab

Releases
Date Semver
DBLab Engine v4.0.1

v4.0.1

Improvements and Fixes

  • Fixed date formatting errors with invalid (empty) dates that could cause crashes on the branches page !1049
  • Implemented proper cleanup for clone datasets on destroy operations !1052
  • Enhanced branch metadata verification after snapshot creation !1051

Internal

  • Removed swagger-ui deployment !1047
  • Updated UI package versions and fixed shared package npm release !1053
  • Updated dependencies to address security vulnerabilities !1050
  • Updated LICENSE file year !1048

Full diff

Diff between versions 4.0.0 and 4.0.1

2 months ago
Download
DBLab Engine v4.0.0

v4.0.0

New features

Improvements and fixes

  • Updated OpenAPI spec – api.dblab.dev !741, !743, !1006
  • Clone counting and restoration cleanup: 9d45ce39
  • Fix “white screen” when opening the clone creation page: !966
  • Reorganizing clone datasets: !977
  • Destroy dependent clones synchronously: !995
  • Refactor API endpoints: !994, !996
  • Fix logs location for idle clone detection: !1012
  • Improve CONTRIBUTING.md !1004, !1005

Internal

  • Clean up and refactor code: !1015
  • Fix integration tests on custom runners: !1010

Full diff

Diff between versions 3.5.0 and 4.0.0

4 months ago
Download
DBLab Engine v4.0.0 RC3

v4.0.0-rc.3

New features

Improvements and fixes

Internal

Full diff

Diff between versions: https://gitlab.com/postgres-ai/database-lab/-/compare/master...dle-4-0

4 months ago
Download
DBLab Engine v3.5.0

v3.5.0

New features

Improvements and fixes

Internal

How to get help

To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.

Provide feedback, contribute

Any feedback will be greatly appreciated:

If you want to help the project grow, consider various ways of contributing:

Full diff

Diff between versions v3.4.0 and v3.5.0: https://github.com/postgres-ai/database-lab-engine/compare/v3.4.0...v3.5.0

Where to start / get help

Provide feedback, contribute

We greatly value your feedback. Connect with us via:

Additional resources where you can get insights about DBLab and Postgres:

Interested in giving back to the project? Here’s how you can make an impact:

almost 2 years ago
Download
DBLab Engine v3.4.0

v3.4.0

In this update:

New name: DBLab Engine

The new name for the Database Lab Engine is “DBLab Engine”. Updates are currently underway across our materials to reflect this change. To align with this change, we have introduced specific domains for the product: dblab.dev and dblab.sh. For ease of access, we have established the following short URLs:

New contributors

🙇 Many thanks to our first-time contributors: @mbobin, @TheLiamGuy, @gromsterus, @ibash, @byx01, @davidsvenson.

By the numbers

Thank you for the support!

New DBLab SE installer

We’ve expanded the installation options for DBLab SE. Besides its presence in the AWS Marketplace, you can now seamlessly install DBLab SE directly from the Postgres.ai Console. This setup is entirely automated and can be used anywhere:

  • For those who have existing machines, we support the “BYOM” (Bring Your Own Machine) method
  • If you’re utilizing AWS, GCP, DigitalOcean, or Hetzner Cloud, the installer handles resource provisioning—including VMs, disks, and more—automatically.

Check out the docs.

Migration notes

For those considering an upgrade, we recommend executing a fresh installation of DBLab. At present, we don’t have automation in place for upgrades. If you need to update an existing DBLab installation, it’s essential to stop and remove the current containers. Then, initiate a fresh start of the DBLab container using the 3.4.0 tag. More details can be found in the docs. Before initiation, ensure you’ve adjusted the DBLab configuration as outlined below.

Postgres images

There is a new method for running clones with customized port and socket directories. Please note that this is NOT backward compatible with previous versions of the Postgres images that DBLab utilized. It is crucial to ensure the dockerImage options utilize tags with the -0.3.0 suffix or later, such as postgresai/extended-postgres:15-0.3.0 (details can be found in this MR).

Global telemetry

The global.telemetry subsection has been deprecated and removed. The control to activate/deactivate telemetry has been relocated to the platform section:

platform:
  url: "https://postgres.ai/api/general"
  enableTelemetry: true

Platform

From version v3.4.0 onward, the platform section is enabled by default.

If either the accessToken or orgKey keys aren’t empty, ensure the url parameter is specified; otherwise, the DBLab instance will not initialize. By default, neither accessToken nor orgKey are defined when in case of self-managed installations.

platform:
  url: "https://postgres.ai/api/general"
  enableTelemetry: true
#  orgKey: "org_key"
#  accessToken: "platform_access_token"

Docker mounts

Please omit the following Docker mount directives during container startup, as they are no longer necessary:

    --volume /sys/kernel/debug:/sys/kernel/debug:rw \
    --volume /lib/modules:/lib/modules:ro \
    --volume /proc:/host_proc:ro \

Changelog

New features

Improvements and fixes

Internal changes

Where to start / get help

Provide feedback, contribute

We greatly value your feedback. Connect with us via:

Additional resources where you can hear more information about DBLab and Postgres:

Interested in giving back to the project? Here’s how you can make an impact:

Diff between versions v3.3.0 and v3.4.0: https://github.com/postgres-ai/database-lab-engine/compare/v3.3.0...v3.4.0

about 2 years ago
Download
Database Lab Engine v3.3.0

v3.3.0

New features

Improvements and fixes

How to get help

To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.

Provide feedback, contribute

Any feedback will be greatly appreciated:

If you want to help the project grow, consider various ways of contributing:

Diff between versions v3.2.0 and v3.3.0: https://github.com/postgres-ai/database-lab-engine/compare/v3.2.0...v3.3.0

almost 3 years ago
Download
Database Lab Engine v3.2.0

v3.2.0

Release v3.2.0 delivers a good number of new features and improvements. DLE UI is extended with two new tabs, “Logs” and “Configuration”, to improve the user experience when configuring or reconfiguring DLE for managed Postgres services such as AWS RDS and Google Cloud SQL:

This allows users to see PGDATA initialization errors when setting up DLE, then adjust the configuration and retry initialization right in the UI – without any need to connect to the DLE machine using SSH.

The demo DLE instance can be found here: https://demo.aws.postgres.ai:446/instance (token: demo_token).

Postgres 15 is already supported!

PostgreSQL 15 is already supported by DLE and included in the list of default images, with many extensions shipped with it: https://postgres.ai/docs/database-lab/supported-databases.

AWS Marketplace – the easiest way to get DLE for any Postgres database

DLE in AWS Marketplace has now General Availability (GA) status, it has already been updated to version 3.2.0: https://aws.amazon.com/marketplace/pp/prodview-wlmm2satykuec, offering the Standard license, with support, embedded monitoring, simplified installation, and access to a wider variety of supported Postgres images (including extensions not available by default). An instance with DLE delivers “dev/test databases as a service” and levels up all dev&test processes. For example, on an r5.xlarge instance (4 vCPUs, 32 GiB RAM) with 200 GiB disk space, it is possible to run up to 30 database clones simultaneously (and even more, if shared_buffers is adjusted to lower values) spending as low as $360 per month – less than for a single regular RDS clone of the same size.

New contributor

🙇 The development team greets a new contributor: @Adrinlol. This release wouldn’t be possible without other contributors: @agneum, @vitabaks, @DmitryFomin1, @ane4ka, @NikolayS, @denis256, @yuridevx.

What’s new

New features

Improvements and fixes

  • Redesign the Retrieval service: !549
  • Mounting of socket directory for non-guest virtualization roles: !557 (@denis256)
  • Add a link to the guide “PostgreSQL configuration in clones”: !559
  • Add session manager to CF template: !560
  • Add maintenance_work_mem for Logical mode: !567
  • Report the type of DLE subscription in UI: !563
  • Detect if DLE is running in a Docker container (this resolves issues of running DLE on KVM): !571, !603
  • Apply the configuration and then start Postgres on the restore stage, to reduce the number of restarts needed: !518, !547 (@yuridevx), !558, !572
  • Fix the creation of a restricted user: !586
  • Build CE UI Docker image after merging into the master branch: !590
  • Adjust container readiness health checks: !597, !602
  • Remove deprecated eslint analyzers: !605
  • Increase the number of parallel jobs in logical mode from 2/2 to 4/4 – usually, many more CPU cores are available: !611

Diff showing all changes

Diff between versions v3.1.2 and v3.2.0: https://github.com/postgres-ai/database-lab-engine/compare/v3.1.2...v3.2.0.

Migration from DLE v3.1.x and configuration notes

There are no special actions that are required to migrate from DLE 3.1.x to 3.2.0: you need to stop the DLE container, change the image, and start a new one using the new image. All snapshots should remain available, and if, at stopping time, some clones were present, they should be automatically recreated by DLE.

Note: if you have an older version, check the previous release notes as well: https://gitlab.com/postgres-ai/database-lab/-/releases.

Configuration directory

The config directory, by default, is ~/.dblab/engine/configs and the corresponding mounted volume must be writable:

  ...
  --volume ~/.dblab/engine/configs:/home/dblab/configs
  ...

Logs directory

Make sure that ~/.dblab/engine/logs is mounted as a volume to the DLE container:

  ...
  --volume ~/.dblab/engine/configs:/home/dblab/configs \
  --volume ~/.dblab/engine/meta:/home/dblab/meta \
  --volume ~/.dblab/engine/logs:/home/dblab/logs \
  ...

Alternatively, you can mount the entire directory (including configs and meta – but make sure that these subdirectories exist, DLE doesn’t create them yet):

  ...
  --volume ~/.dblab/engine:/home/dblab
  ...

Query preprocessing

The queryPreprocessing section supports inline SQL now. Examples can be found here: !576#examples

Configuration UI

Because the configuration can be viewed and edited in the UI, make sure:

  • secret values are masked (e.g., verificationToken, Platform accessToken, you can find all masking rules here: !608#masking-rules)
  • DLE configuration matches the reference configuration files in structure and naming; it is not recommended to have custom names and sections

Where to get help

To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.

Provide feedback, contribute

Any feedback will be greatly appreciated:

If you want to help the project grow, consider various ways of contributing:

about 3 years ago
Download
v3.1.2

v3.1.2

Database Lab Engine v3.1.2

DLE v3.1.2 delivers multiple bug fixes and improvements.

🙇 The development team greets new contributors: @yuridevx, @denis256.

New features and logic adjustments

Improvements and fixes

Contributor’s experience

How to get help

To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.

Provide feedback, contribute

Any feedback will be greatly appreciated:

If you want to help the project grow, consider various ways of contributing:

Diff between versions v3.1.1 and v3.1.2: https://github.com/postgres-ai/database-lab-engine/compare/v3.1.1...v3.1.2

over 3 years ago
Download
v3.1.1

v3.1.1

Database Lab Engine v3.1.1

DLE v3.1.1 delivers multiple bug fixes and improvements.

The development team greets a new contributor: @Alexand.

Improvements and fixes

How to get help

To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.

Provide feedback, contribute

Any feedback will be greatly appreciated:

If you want to help the project grow, consider various ways of contributing:

Diff between versions v3.1.0 and v3.1.1: https://github.com/postgres-ai/database-lab-engine/compare/v3.1.0...v3.1.1

over 3 years ago
Download
Database Lab Engine v3.1.0

v3.1.0

Database Lab Engine v3.1.0

DLE 3.1 has many improvements and several new features, such as native support of pgBackRest and the ability to configure the timezone when using CLI.

In this release, the development team has focused on the Database Lab Engine community, making it easier to get help or contribute. The team greets all new contributors: @Nikolay Devxx, @asotolongo, @Tanya301, @denis-boost, @pietervincken, @ane4ka

Community news

The README.md has been significantly improved. It now has four translations, with more coming in the future. Many thanks to our README translation contributors! It helps engineers worldwide understand DLE concepts and improve the development experience by building powerful dev&test environments.

Migration notes

⚠️ Prior to version 3.1, the default ZFS version DLE’s Docker images were using was 0.8.x. For DLE 3.1, the default Docker image (postgresai/dblab-server:3.1.0) uses ZFS 2.1.2. If you need to upgrade DLE from a previous version that worked with a ZFS pool 0.8.x (default for previous DLE versions), consider the following options:

  • Option 1: upgrade your system to use ZFS 2.1, optionally upgrade your pool (zpool upgrade dblab_pool), and then upgrade DLE to use the default image postgresai/dblab-server:3.1.0
  • Option 2: postpone the ZFS upgrade, stay on ZFS 0.8, and upgrade DLE to version 3.1 using a special image, postgresai/dblab-server:3.1.0-zfs0.8

Changelog

New features

Improvements and fixes

Internal

How to get help

To get help, reach out to the Postgres.ai team and the growing community of Database Lab users and contributors: https://postgres.ai/contact.

Provide feedback, contribute

Any feedback will be greatly appreciated:

If you want to help the project grow, consider various ways of contributing:

Diff between versions v3.0.0 and v3.1.0-rc.1: https://github.com/postgres-ai/database-lab-engine/compare/v3.0.0...v3.1.0-rc.1

over 3 years ago
Download