Databricks spent most of the Summit keynote telling us LTAP means "no more pipelines, no more ETL, one copy of data." Fine. I've heard "no more ETL" enough times to be suspicious of it on reflex. But I got curious about the one part nobody really spells out in the press releases: if your app is writing plain Postgres rows, how does that same data show up in Iceberg as columns, fast enough to query, without a pipeline you can actually see? So I went reading. Here's the mechanism as best I can piece it together.

First, why this is even a problem. Postgres stores data by row. Everything about one record sits together on disk, which is exactly what you want for "give me user 48213 and update their balance", since you touch one row and you're done. Analytics wants the opposite. "Average order value over the last 90 days" only needs one column out of forty, but in a row store you still drag every row (and every other column) off disk to get at it. Columnar formats like Parquet flip the layout so each column is stored together:

Row store (Postgres):

[id=1, name=Ana, amount=50]

[id=2, name=Ben, amount=80]

[id=3, name=Cy, amount=20]

Column store (Parquet / Iceberg):

id: 1, 2, 3

name: Ana, Ben, Cy

amount: 50, 80, 20

Now "sum the amounts" reads one tidy contiguous list and skips everything else. It also compresses far better, because similar values end up sitting next to each other. Old Lakebase basically kept Postgres data in Postgres format on object storage, so you still needed a conversion step before the analytical engines could do anything useful with it. LTAP's whole pitch is killing that step.

The thing actually doing the work is Moonlink, a component from the Mooncake team Databricks bought last year. It's a replication engine (written in Rust, for what it's worth). It taps Postgres's logical replication stream, the same change feed Postgres already emits for replicas. Every insert, update and delete flows out of that feed. Moonlink consumes it and mirrors the changes into Iceberg, but rewritten as columns, with sub-second lag.

The detail I found genuinely clever is where the row-to-column conversion happens. Object storage (S3 and friends) is slow: response times in the seconds, way too slow to serve actual transactions. So Postgres keeps a fast caching tier in front of it, and that tier usually has spare CPU sitting idle. Databricks does the transcode right there, on that idle CPU, before the data ever lands in object storage. And because going row-to-column compresses something like 10x, you've also shrunk what you have to push down to S3 in the first place. You're not paying for a separate conversion job later; you're paying with cycles that were going to waste anyway, and you ship less data for the trouble. (Reynold Xin walked through this in a VentureBeat interview if you want the source.)

The other half is the freshness trick, because there's normally an annoying tradeoff here. If Moonlink wrote every single change straight into a new Iceberg file, you'd get freshness but also a blizzard of tiny files and constant metadata commits, which is miserable to live with. If it batches writes up to be efficient instead, your analytics goes stale. Moonlink dodges this with what they call union reads. Newly arrived rows sit in an in-memory columnar buffer (Arrow). A query then reads the committed Parquet files on object storage, that in-memory buffer, and any pending updates or deletes, all stitched together as one logical table. That means an analytical query can see data that hasn't even been written into an Iceberg snapshot yet. That's how they claim sub-second freshness without drowning in small files.

One thing worth being clear-eyed about: this isn't HTAP in the old "one engine does both" sense, no matter how the slides read. Postgres is still the transactional engine; Spark/Photon and the new Reyden engine are still the analytical ones. A commit in Postgres is not running your analytical query inside the same transaction. What's actually unified is the storage. One logical copy of the data, one write path (Postgres to Moonlink to Iceberg), and several read paths sitting on top. "One copy for everything", not "one engine for everything". Which, honestly, is probably the more achievable version of the dream anyway.

Caveat: LTAP was just announced and is rolling out as part of Lakebase, so most of this is stitched together from the announcement, interviews, and a couple of good technical breakdowns rather than me running it in prod. If anyone's got it live in their workspace and I've got a detail wrong, please correct me.

TL;DR: Your app writes rows to Postgres like normal. Moonlink tails Postgres's replication stream, converts those changes to columnar Parquet on idle CPU in the caching tier (which also compresses around 10x before anything hits S3), and writes them into Iceberg. Queries read the committed columnar files plus an in-memory buffer of brand-new rows together (union reads), so analytics stays fresh within a second. Separate engines, one shared copy of the data, not classic single-engine HTAP.

I’m implementing Lakebase branching strategy to improve development experience and reduce costs for our dev/staging env.

Current setup creates new database branch for each git branch via githook on our dev database (”each dev gets their own feature database”). There is also similar workflow for each PR against our staging database to run the migrations and tests.

Curious to hear how others are using branching and what are the experiences?

Making Postgres materialized view refreshes O(delta) instead of O(total)

Many developers assume PostgreSQL materialized views handle incremental updates out of the box. They do not. If you have a materialized view with millions of rows and a single underlying record changes, both native refresh options process the entire dataset.

Because of this limitation, anyone needing immediate maintenance or working with large datasets must abandon materialized views entirely. The standard workaround is to manually maintain a standard table using custom database triggers or application logic.

I've been working on a patch to fix this. It adds an optional WHERE clause to the REFRESH MATERIALIZED VIEW command, letting you scope a refresh to exactly the rows that changed. The patch is currently under review on the pgsql-hackers mailing list.

This approach requires two things. First, the materialized view must have a unique index (the same requirement as REFRESH MATERIALIZED VIEW ... CONCURRENTLY). Second, the view's underlying query must allow the planner to push down the WHERE predicate. If the base query contains opaque boundaries like materialized CTEs, certain window functions, or un-inlineable functions, the planner cannot push the predicate to the base tables. The engine will instead execute a full recalculation and filter the results post-hoc, which defeats the performance benefit.

It allows for targeted, partial refreshes. For example:

REFRESH MATERIALIZED VIEW invoice_totals WHERE invoice_id = 42;

Instead of processing the entire dataset, the optimizer pushes the predicate down to the base tables. This makes the refresh proportional to the data changed rather than the total size of the view.

This patch only implements the syntax and internal execution logic for partial refreshes. How you derive the parameters for the WHERE clause is up to the implementer.

This enables multiple incremental maintenance patterns. Broadly, these fall into the two established categories of view maintenance theory: Immediate and Deferred. The trigger-based implementations below demonstrate zero-touch automation, but simpler orchestration methods like cron jobs querying timestamp columns work just as well.

Immediate View Maintenance: Synchronous statement-level triggers

In immediate maintenance, the materialized view is updated in the exact same transaction that modifies the underlying base tables.

If you need the view strictly current by the time the writing transaction commits, drive these immediate refreshes from statement-level triggers using transition tables. Transition tables expose the exact rows modified by a statement as a queryable relation. You extract only the affected keys and pass them to the refresh.

A single function handles all three operations. PostgreSQL requires single-event triggers when transition tables are involved, so each source table gets three triggers sharing one function.

```sql CREATE OR REPLACE FUNCTION refresh_invoice_totals() RETURNS trigger LANGUAGE plpgsql AS $$ DECLARE affected_ids int[]; BEGIN IF TG_OP = 'INSERT' THEN SELECT array_agg(DISTINCT invoice_id) INTO affected_ids FROM new_rows; ELSIF TG_OP = 'UPDATE' THEN SELECT array_agg(DISTINCT invoice_id) INTO affected_ids FROM (SELECT invoice_id FROM new_rows UNION SELECT invoice_id FROM old_rows) combined; ELSIF TG_OP = 'DELETE' THEN SELECT array_agg(DISTINCT invoice_id) INTO affected_ids FROM old_rows; END IF;

IF affected_ids IS NOT NULL THEN EXECUTE 'REFRESH MATERIALIZED VIEW invoice_totals WHERE invoice_id = ANY($1)' USING affected_ids; END IF;

RETURN NULL; END; $$;

-- Triggers for invoice_lines CREATE TRIGGER refresh_on_line_insert AFTER INSERT ON invoice_lines REFERENCING NEW TABLE AS new_rows FOR EACH STATEMENT EXECUTE FUNCTION refresh_invoice_totals();

CREATE TRIGGER refresh_on_line_update AFTER UPDATE ON invoice_lines REFERENCING NEW TABLE AS new_rows OLD TABLE AS old_rows FOR EACH STATEMENT EXECUTE FUNCTION refresh_invoice_totals();

CREATE TRIGGER refresh_on_line_delete AFTER DELETE ON invoice_lines REFERENCING OLD TABLE AS old_rows FOR EACH STATEMENT EXECUTE FUNCTION refresh_invoice_totals();

-- Triggers for invoices CREATE TRIGGER refresh_on_invoice_insert AFTER INSERT ON invoices REFERENCING NEW TABLE AS new_rows FOR EACH STATEMENT EXECUTE FUNCTION refresh_invoice_totals();

CREATE TRIGGER refresh_on_invoice_update AFTER UPDATE ON invoices REFERENCING NEW TABLE AS new_rows OLD TABLE AS old_rows FOR EACH STATEMENT EXECUTE FUNCTION refresh_invoice_totals();

CREATE TRIGGER refresh_on_invoice_delete AFTER DELETE ON invoices REFERENCING OLD TABLE AS old_rows FOR EACH STATEMENT EXECUTE FUNCTION refresh_invoice_totals(); ```

By adding these triggers to both the invoice_lines and invoices tables, changes to both line items and headers will immediately and synchronously propagate to the materialized view.

Deferred View Maintenance: Asynchronous staging tables and pg_cron

In deferred maintenance, the view update happens after the transaction commits, often periodically.

If write latency matters and you can tolerate bounded staleness, decouple writes from refreshes entirely. A lightweight row-level trigger accumulates affected keys into an unlogged staging table. A single trigger function handles INSERT, UPDATE, and DELETE. A pg_cron job drains the queue on a schedule, refreshes the accumulated subset, and clears the table.

```sql CREATE UNLOGGED TABLE invoice_refresh_queue ( invoice_id int PRIMARY KEY );

CREATE OR REPLACE FUNCTION queue_invoice_refresh() RETURNS trigger LANGUAGE plpgsql AS $$ BEGIN INSERT INTO invoice_refresh_queue (invoice_id) VALUES (CASE WHEN TG_OP = 'DELETE' THEN OLD.invoice_id ELSE NEW.invoice_id END) ON CONFLICT DO NOTHING; RETURN NULL; END; $$;

CREATE TRIGGER queue_on_line_change AFTER INSERT OR UPDATE OR DELETE ON invoice_lines FOR EACH ROW EXECUTE FUNCTION queue_invoice_refresh();

CREATE TRIGGER queue_on_invoice_change AFTER INSERT OR UPDATE OR DELETE ON invoices FOR EACH ROW EXECUTE FUNCTION queue_invoice_refresh();

CREATE OR REPLACE FUNCTION drain_invoice_refresh_queue() RETURNS void LANGUAGE plpgsql AS $$ DECLARE queued_ids int[]; BEGIN WITH deleted AS ( DELETE FROM invoice_refresh_queue RETURNING invoice_id ) SELECT array_agg(invoice_id) INTO queued_ids FROM deleted;

IF queued_ids IS NOT NULL THEN EXECUTE 'REFRESH MATERIALIZED VIEW invoice_totals WHERE invoice_id = ANY($1)' USING queued_ids; END IF; END; $$;

SELECT cron.schedule( 'drain-invoice-refresh', '* * * * *', 'SELECT drain_invoice_refresh_queue()' ); ```

The trigger cost is a single-row insert into an unlogged table per write. The cron job batches everything accumulated since the last run into a single refresh statement. If nothing changed, the drain exits immediately. This is similar (if you squint) to one of the methods Oracle uses for materialized view incremental refreshes, implemented in standard SQL.

If you want to avoid triggers entirely and your schema tracks modification timestamps, you can implement deferred maintenance using a watermark table. A scheduled job retrieves the last execution timestamp, queries the base tables for records modified since that watermark, and passes those IDs to the refresh command: REFRESH MATERIALIZED VIEW invoice_totals WHERE invoice_id = ANY($1).

The tradeoffs are straightforward. Immediate maintenance gives you absolute consistency within the writing transaction at the cost of added write latency. Deferred maintenance minimizes write overhead at the cost of a staleness window. Watermark-based deferred maintenance further reduces complexity, but requires standard audit columns on all base tables.

Alternatives and Prior Art

Extensions like pg_ivm and TimescaleDB exist, but they serve specific niches. TimescaleDB targets continuous time-series aggregation. Many production materialized views do not aggregate data at all. They are complex, non-aggregate queries used to resolve operational state or entity eligibility. pg_ivm provides immediate view maintenance but imposes strict limitations on the supported SQL syntax and query structures. Implementing targeted refreshes directly in the core engine provides a general-purpose mechanism that respects standard query planner semantics without the overhead or query restrictions of an extension.

Getting the concurrency model right

This took a few iterations. My original implementation used a naive two-step strategy: a DELETE query followed by an UPSERT. This failed to handle locking properly. The DELETE step immediately destroyed the physical row locks. In the gap between the delete and the upsert, concurrent transactions could insert colliding logical rows, leading to database inconsistencies and constraint violations.

I then tried using transaction-level advisory locks to bridge that consistency gap. Testing revealed that this approach fails at scale, hitting max_locks_per_transaction limits and breaking down during bulk operations.

I ended up rewriting the non-concurrent path to use a two-step SPI execution strategy:

1. It executes a SELECT FOR UPDATE to lock the existing rows matching the predicate. This safely serializes concurrent partial refreshes on overlapping rows.
2. It executes a single CTE that evaluates the underlying query, upserts the results into the materialized view, and deletes rows that no longer match the predicate via an anti-join.

I also added a session-level cache for the prepared SPI plans to avoid recompilation overhead on frequent trigger-based refreshes.

Future work

The immediate and deferred patterns shown above work today but require manual setup: writing trigger functions, wiring them to every source table, and in the deferred case, creating staging tables and scheduling cron jobs. A natural next step is pushing this ceremony into the engine itself.

Testing it out

To test this out yourself, you'll need to compile Postgres using the patch linked in the mailing list thread below. I've put together a small test harness in a single Gist (containing both setup_demo.sql and demo.sql).

Once your patched instance is running, you can execute the test harness directly via psql.

First, run the setup script to create the schema and generate 10,000 sample invoices to give us a baseline: bash psql -d your_database_name -f setup_demo.sql

Then, run the demo file. This acts as an interactive tutorial, walking through both the immediate (statement-level triggers) and deferred (staging table) patterns with test cases that prove the partial refreshes work: bash psql -d your_database_name -f demo.sql

The full thread, patch, and pgbench results are on the pgsql-hackers mailing list. I would appreciate any and all feedback!

EDIT: I've taken some of the feedback from below and made some updates to the article above, which hopefully leaves people less confused going forward. Thank you all for the discussion.

Hi all,

Over the past year, the common refrain in the AI community has been "Postgres isn't built for vector search; you need a dedicated vector database."

Having run PostgreSQL in production for years, I was skeptical. Dedicated vector DBs introduce another stateful service to back up, monitor, and pay for, while breaking ACID guarantees across relational metadata and vector embeddings.

We built Knowledge Fabric as an open-source proof that modern PostgreSQL (16+) handles production-scale RAG workloads cleanly:

1. HNSW Vector Indexing: Using `pgvector` with HNSW (`vector_cosine_ops`, `m=16, ef_construction=64`), query times remain sub-10ms even on millions of vectors.
2. True Hybrid Search in 1 Query Engine: Instead of querying Elasticsearch for BM25 and Pinecone for cosine, then stitching them together in Python, we run `tsvector` full-text search and `pgvector` dense search in PostgreSQL and merge them using Reciprocal Rank Fusion:```sql-- Score = 1 / (60 + rank_lexical) + 1 / (60 + rank_vector)```
3. Database-Enforced Multi-Tenancy (RLS): For compliance (HIPAA / SOC 2), relying on application-level `WHERE tenant_id = 'xyz'` is vulnerable to developer oversight. We added opt-in PostgreSQL Row-Level Security:

​

CREATE POLICY tenant_isolation_policy ON chunks FOR ALL USING (tenant_id = current_setting('app.tenant_id', true));

If a query fails to set `app.tenant_id`, the database returns zero rows. Cross-tenant leakage is physically impossible.

4. Declarative Partitioning for 50M+ Chunks: By partitioning the `chunks` table `BY LIST (tenant_id)`, multi-tenant queries benefit from partition pruning, scanning only the tenant's localized HNSW index.

Full code and SQL migration schemas are open source:

https://github.com/sagarv48/knowledge-fabric

Curious how other DBAs and architects are handling `pgvector` memory tuning (`maintenance_work_mem`) and HNSW index build times on large datasets.

I added the TurboQuant algorithm published by Google to pgvector as part of my discovery and learning process with RAG systems. Just this past weekend, I ran a test with the 100M row Wikipedia dataset from Cohere where I observed a 10x reduction in index size relative to HNSW. I figure with the direction RAM and storage prices have been going, we could use some more ways to save space!

• 76 PostgreSQL tools — query execution, schema inspection, DDL operations, batch operations, monitoring, maintenance, replication, transactions, and more
• PostgreSQL documentation-compliant — all queries verified against official PG docs (v16-18). Uses correct view/column names across PG versions with graceful fallbacks
• Dual-protocol transport — TCP (port 3000) and HTTP/2 (port 3001) for flexibility
• Sub-10ms latency — optimized for interactive AI workflows
• Production-grade — connection pooling, health checks, input validation, SQL injection prevention
• Stateless HTTP — each request is independent (no transaction state across requests)

https://github.com/corporatepiyush/mcp-pg-rust

A while back I had to integrate data from a third-party REST API into a Postgres-backed app. My solution at the time was a cron job that periodically fetched the API, parsed the response, and shoved it into the database. It worked. It was also annoying to maintain and broke in creative ways. Months later I discovered that Postgres could have queried that API directly (and I felt a bit dumb lol).

The feature is called Foreign Data Wrappers, and it's been in Postgres for years. The idea: you create a virtual foreign table that maps to an external data source, then you query it with plain SQL. JOINs, WHERE clauses, INSERTs from SELECT, the whole deal.

Here's what I've been using it for:

CSV files without the import dance 

Postgres ships with file_fdw. You point it at a CSV, define the columns, and it's a queryable table. You can JOIN it with your real tables or cherry-pick rows to INSERT into a permanent table. No more writing throwaway Python scripts to parse CSVs. One catch: file_fdw is read-only, so no writing back to the file.

Querying a remote Postgres database 

postgres_fdw is also built-in. You set up a foreign server, map a user, create the foreign table, and suddenly you can query (and even UPDATE) another Postgres instance from your local one. Handy for migrations or cross-database reporting. Setting up the user mapping with credentials in plain SQL feels a bit rough, but it gets the job done.

Talking to MongoDB (or any NoSQL store) 

This is where it gets fun. With Multicorn (a Python library) you can write your own FDW for pretty much anything. You define a Python class, implement an execute method that translates SQL qualifiers into queries for your target data source, and Postgres handles the rest. There are also ready-made FDWs for MongoDB, ElasticSearch, Redis, and others if you don't want to roll your own ;)

REST APIs as tables 

Same principle with Multicorn. You write a wrapper class that turns WHERE clauses into API query parameters, hits the endpoint, and yields rows back to Postgres. I used the Magic: The Gathering API as a test case, nothing mission-critical, but the pattern translates to any REST endpoint. For authenticated APIs you just add headers or tokens in the Python code.

That said, it's not all smooth sailing. JOINs between foreign tables and local ones can get slow, especially with large external datasets. Also, debugging a misbehaving custom FDW is... not fun lol. And writing credentials in plain SQL for user mappings still makes me wince every time.

For those of you already running FDWs in production, how do you handle the performance tradeoff? Curious what strategies people have settled on ;)

Dear all, here are some updates for the open source tool I've created for PostgreSQL.

What's new:

Table view

• Smart forms with related data section to navigate the full relational context without leaving the record
• Add linked data columns, aggregate, sort, render as inline charts

Schema diagram

• Explore tables and foreign keys with custom color modes and table icons

AI Assistant

In addition to the usual "look at my schema, analyse data, create a dashboard" it allows:

• Per-chat access control - scope each conversation to specific tables, columns, rows, MCP tools and configurations
• Progressive discovery - schema and tools are loaded on demand, keeping context lean in large environments
• Agentic workflows - (my favourite) describe a data task and get back a TypeScript orchestration that runs in an isolated container with defined permissions. Inspect, tweak, re-run, and view live logs
• Local service integrations - Speech-to-text (Faster-Whisper), web search (SearXNG), document extraction (Docling)

Command palette

• Ctrl+K to find and jump to the specific section/feature without having to remember buttons and menus

Website: https://prostgles.com/

Online demo (limited): https://playground.prostgles.com/

pg_durable is a Microsoft developed PostgreSQL extension that integrates fault-tolerant, long-running execution directly into the database.

To understand why this is useful, think about what normally happens if you run a long, complex database job and the server crashes or your connection drops; you lose your progress, and you have to start all over again.

https://www.i-programmer.info/news/80-java/18984-pgdurable-durable-sql-functions-and-orchestration-for-postgresql.html

I just shipped bitemporal provenance for TypeGraph, my open-source graphs-on-SQL library. Three pieces, usable independently but most powerful together:

• Valid time: when a fact was true in the world (an invoice's effective date, a role grant's window).
• Recorded/system time: when the system captured that fact (what you knew, as of a commit instant; the SQL:2011 FOR SYSTEM_TIME / Datomic system-time axis).
• Provenance: why the system still believes a derived fact, and what happens downstream when a source it depended on turns out to be wrong.

Derived facts are the annoying case that surfaces the issue(s) these primitives solve. For example, a Vulnerability node exists because a scanner and a vendor advisory both pointed at it. The graph concluded it; nobody asserted it directly.

ScannerSource ──┐ ├──▶ Vulnerability (CVE-2026-1234, libvector) VendorSource ──┘

So when the scanner turns out to be garbage, you can't treat retracting it as a delete. The vendor might still back that vulnerability. The scanner might have been the only thing propping up a bunch of other facts. You want the graph to sort out which.

What you want: retract a source and it recomputes which derived facts still have grounded support. Retract the vendor too and the vulnerability finally goes non-current, and a "block the deploy" decision sitting on top of it goes with it.

The behavior, then the theory

A fact stays believed while it has at least one justification whose premises are all still supported. Premises bottom out at sources. Retract a source and every justification that leaned on it stops counting; a fact loses currency only once it runs out of surviving justifications.

```typescript const provenance = createRetractionCapability(store, { source: { kinds: ["ScannerSource", "VendorSource"] }, justification: { kind: "Justification" }, fact: { kinds: ["Vulnerability", "DeployDecision"] }, premiseOf: { kind: "premiseOf" }, derives: { kind: "derives" }, });

const report = await provenance.retract({ kind: "VendorSource", id: vendorId }); // report.died: facts that lost all grounded support // report.survivedVia: facts that still have an alternate justification ```

This is modeled on truth-maintenance systems. The storage follows the JTMS shape (Doyle 1979, "A Truth Maintenance System"): AND-justifications over premises, sources at the bottom, a fact in the well-founded support set only if some justification has all its premises supported. I use the monotonic, inlist-only fragment, so this is the easy part of Doyle's system; the hard part, non-monotonic belief revision, isn't here. The question retract actually answers, "which facts survive because an alternate justification still holds," is the ATMS question (de Kleer 1986): which combinations of sources hold each fact up. So it's JTMS-shaped storage with an ATMS-flavored query.

Retraction is a normal write, so you get replay for free

Retraction doesn't hard-delete. It recomputes support and flips unsupported facts to non-current, leaving the justification edges in place so you can still see why something used to be believed. Because that write lands on TypeGraph's recorded-time (system-time) substrate, you can replay the belief transition:

```typescript const before = await store.recordedNow(); await provenance.retract(badSource); const after = await store.recordedNow();

await store.asOfRecorded(before).nodes.Vulnerability.getById(id); // believed await store.asOfRecorded(after).nodes.Vulnerability.getById(id); // not current ```

TypeGraph tracks both temporal axes as explicit read lenses, valid time ("when true in the world") and recorded time ("when the database learned it"), and because they're lenses they compose:

typescript store.asOf(validTime).asOfRecorded(recordedTime)

Architecture

No engine-native temporal tables. Postgres needs an extension for system-versioning and SQLite has nothing, so TypeGraph stores history explicitly and reconstructs point-in-time views in the query compiler. That's why one implementation runs on both backends.

Limits

• Only TypeGraph-managed writes are captured. Raw SQL bypasses it; this isn't a database-level CDC/audit layer.
• No backfill. Enable history on a fresh graph.
• Point-in-time reads reconstruct from history relations, so they're slower than current-state reads. It's an audit tool, keep it off hot paths.
• Per-write overhead runs ~2.5–6x unless you batch writes in one transaction, where it drops to ~1–1.5x.

A naming note

My asOf is valid time, the reverse of SQL:2011 FOR SYSTEM_TIME AS OF and Datomic (d/as-of db t), where a bare as-of is system time. Valid-time reads are the common case here so they took the short name; system time is asOfRecorded.

I'd love to compare with other systems that handle provenance retraction, or truth maintenance generally, modeled directly on ordinary SQL tables instead of a dedicated reasoning engine. There's plenty of JTMS/ATMS literature but not much on mapping it onto relational storage. Pointers welcome.

GitHub: https://github.com/nicia-ai/typegraph Docs: https://typegraph.dev/provenance

Examples: https://typegraph.dev/examples/provenance-retraction/ https://typegraph.dev/examples/bitemporal-time-travel/

Many years ago I added the same capability to MS SQL Server Studio because I run a lot of long running queries, data warehouse builds, etc. Nowadays, I only use postgres but still run things that take 30mins to 5 hours. The issue is I have to constantly keep checking if the query is complete. So Im building the same thing into pgadmin that I built in SSMS, a set of options to let you specify how you want to be alerted when a query is finished running. Flash window, Play Sound, Send Email. Send email includes the query and the amount of time to complete.

I dont know that I am going to submit this to be an actual feature unless I get an overwhelming response to do so. I am sure the pgadmin gods will have plenty to say about how it gets implemented.

looking for your feedback on whether this would be useful for you

Created an extenstion for it, allow deep merge etc. It's faster than doing it using SQL queries and works quite well for us ( used in production on our events base backend ).

https://github.com/olivierchatry/pg_jsonb_merge

If you've ever needed to migrate data from a MySQL, SQL Server, or Oracle dump into PostgreSQL, you know the pain. Replaying INSERT statements is slow, pgloader has its quirks, and setting up the source database just to re-export is a hassle.

I built **sql-to-csv** — a CLI tool that converts SQL dump files directly into:

- CSV/TSV files (one per table) ready for `COPY`

- A `schema.sql` with the DDL translated to PostgreSQL types

- A `load.sql` script that runs schema creation + COPY in one command

It handles type conversion automatically (e.g. MySQL `TINYINT(1)` → `BOOLEAN`, SQL Server `UNIQUEIDENTIFIER` → `UUID`, Oracle `NUMBER(10)` → `BIGINT`, etc.) and warns about things it can't convert.

Usage is simple:

```

sql-to-csv dump.sql output/

psql -d mydb -f output/load.sql

```

It auto-detects the source dialect (MySQL, PostgreSQL, SQL Server, Oracle, SQLite) and uses parallel workers to process large dumps fast. A 6GB Wikimedia MySQL dump converts in about 11 seconds.

GitHub: https://github.com/bmamouri/sql-to-csv

Install: `brew tap bmamouri/sql-to-csv && brew install sql-to-csv`