Bookmarks (1307)

How Uber is Evolving Schemaless into a Distributed SQL Database — An update to a hugely popular set of blog posts from 2016 about how Uber built a ‘schemaless’ data store on top of MySQL. Their thinking has evolved somewhat since then. Uber is now working on ‘Docstore’, a multi-model database that can scale horizontally – it still uses MySQL under the hood, though.

Workday - PostgreSQL Database Engineer / Administrator — We’re looking for an engineer to join our growing SaaS company to help revamp our persistence architecture, ensure 24/7 ops, and help us build for the future.

TimescaleDB 2.1 Released — A popular time-series oriented database on top of Postgres. 2.1 adds Postgres 13 support.

Redigo: A Go Client for Redis — Supports all the Redis commands through a Print-like API and also offers connection pooling, helper functions for handling replies, and pipelining support.

pgBadger 11.5 Released: A Fast Postgres Log Analyzer — Get info about queries, running time, server info, and more. 11.5 adds support for anonymization of numeric values for sensitive use cases, as well as more info about session idle time. They’re a bit out of date, but here’s an example of the sort of report it can generate.

EdgeDB 1.0 Beta 1 Unveiled — EdgeDB is a mildly stealthy database we’ve covered a few times in the past few years and which is very gradually approaching its 1.0 release. It’s built on top of Postgres and aims to offer the best parts of NoSQL but with a new set logic based query language aimed to address SQL’s shortcomings.

An Amazon DocumentDB re:Invent 2020 Recap — Now at just over two years old, AWS’s MongoDB-a-like has certainly caused some drama in its lifetime, but continues to innovate and attract new users. These talks range from a deep dive and how to migrate to DocumentDB through to what new features have been added recently.

A Deep Dive Into Roam's Data Structure — Roam is a note-taking tool I’ve seen popular amongst the ‘productivity’ folks on Twitter and a developer has dug into what sort of data model it uses.

Modernizing from PostgreSQL to Serverless with Fauna — It’s pays to note this series of blog posts is by Fauna so there’s natural bias here, but any “why you might move from X to Y” example interests me when it comes to data, and serverless data APIs do offer a lot of benefits for certain use cases. Seeing how they remodel the data is...

Using Postgres to Do a Regex-Based Search Over 10,000 GitHub Repositories — Empirical measurements from experiments in using Postgres to index and search over 10,000 popular GitHub repositories using pg_trgm (trigram matching functions) on a 2019 MacBook Pro.

29 Bite-Sized DynamoDB Best Practices — Bite-sized best practices for DynamoDB modeling, security, performance and design. Almost a checklist, if you will.

Six Common Mistakes for SQL Queries That 'Should Be Working' — I think I’ve been lucky that I haven’t gotten stuck with an SQL query for a prolonged amount of time, but if you have, maybe it’s on one of these tripping points?

👋The What, Why, and How of Time-Series Databases — Time-series databases are the fastest growing database category – but what are they? Get a primer on what time-series data is - complete with real-world examples - when and why “normal” databases aren’t enough, and ways to get started.

Read Consistency with Database Replicas at Shopify — When you’re as big as Shopify, of course you have a ‘Database Connection Management’ team, and here we get to see how they solve issues around lag caused by database replication.

Many Small Queries Are Efficient In SQLite — Making the point that while 200 SQL queries to put together a single Web page could be considered inefficient with many database systems, it’s nothing for SQLite.

Synthetic Data: Generating Realistic User Timestamps in SQL — How to generate synthetic timestamps with SQL that match realistic user behavior data versus purely picking random times.

How Spotify Optimized the 'Largest Dataflow Job Ever' for Wrapped 2020 — Each year the music service Spotify puts together customized reports for every user which highlights the music they were listening to the most often, genres they liked, and similar tidbits. This involves a lot of data, and for Wrapped 2020 they used the Sort Merge Bucket (SMB) join technique to hugely reduce costs...

An Operations-Free, Scalable & Flexible Postgres Alternative — Fauna combines the operational integrity and relational modeling of Postgres with an interface and architecture that fits better with modern app development in the cloud. The goodness of Postgres without its operational bottlenecks - Learn more about Fauna.

European Patent Office Considers an Extension to SQL Patent-Eligible — This is quite a legalistic piece on how the European Patent Office has considered an SQL language extension for modifying collection-valued and scalar valued columns in a single statement to be ‘technical’.

How Materialize and Other Databases Optimize SQL Subqueries — Not something you commonly think about while using subqueries (in my experience) but useful to know what helps database systems handle them better.

Backend Developer - Remote or in Beautiful Norway — Do you have a passion for GraphQL, NodeJS, and message drive distributed architectures? Join our remote-first engineering team.

Querying JSON Data in PostgreSQL — I’ve been working directly with querying JSON directly in Postgres a lot lately and it’s been.. rather pleasant! Nothing mindblowing in this post but it covers the basics if you haven’t dipped a toe into the JSONB column world yet.

The 20 Most-Visited Amazon DynamoDB Documentation Pages — It’s interesting to see AWS reveal information like this. Popular topics include how to work with DynamoDB Local, using global secondary indexes, and working with expressions.

Simulating Latency with SQL / JDBC — “I’ve run across a fun little trick to simulate latency in your development environments when testing some SQL queries..”

Adminer: Database Management in a Single PHP File — A mature project that’s an alternative to phpMyAdmin for managing Postgres, MySQL, SQL Server, and other databases via a Web interface. The main selling point is it’s distributed as a single PHP file. v4.8.0 just dropped. GitHub repo.

rqlite 5.10.0 Released — rqlite is a lightweight, distributed relational database system written in Go that uses SQLite as its storage engine. 5.10.0 is hugely focused on reductions in disk usage.

Obevo: Database Deployment Tool for Enterprise Scale and Complexity — This is for managing and maintaining large collections of database files containing your schemas, procedures, etc. Supports DB2, H2, SQL Server, MongoDB, Oracle, Postgres, Redshift, and more.

RocksDB: A Persistent Key-Value Store for Flash and RAM Storage — It’s been several years since we’ve linked to RocksDB and since then it has become the foundation for numerous database systems. It continues to get frequent updates, such as this week’s v6.15.5 release – it’s certainly worth investigating if you’re planning to build a database of your own.

PostgreSQL 13.2, 12.6, 11.11, 10.16, 9.6.21, and 9.5.25 Released — A suite of updates to Postgres to fix two security vulnerabilities. This is also expected to be the final 9.5 branch release ever.

TinyDB: A Lightweight Document Oriented Database in Pure Python — Essentially a JSON store aimed at small Python apps that need some structured data storage but without an external database server or any SQL involved.

[NEW] Benchmarking the Cloud: AWS vs. GCP vs. Azure — Which cloud has the lowest latency? The best storage performance? The most throughput? This report digs into the tradeoffs each cloud makes - so you can choose the best option for your needs.

Implementing a Generic REDUCE Aggregate Function with SQL — A fun bit of SQL spelunking here for arbitrarily reducing a collection of data.

▶  The AWS re:Invent 2020 Database Breakout Sessions — Amazon has uploaded all of the sessions from its recent re:Invent conference to YouTube, and the database track is buzzing with all sorts of AWS-flavored database talks. Almost no-one will watch them all, but there’s lots of good stuff in here. Some highlights:

Patterns for AWS IoT Time Series Data Ingestion with Amazon Timestream — Timestream is Amazon’s managed time series database. This post covers approaches to using it for high velocity data ingestion from IoT data sources.

Considering Dependent and Independent Pieces of Data — How to use quadrant analysis to find dependent and independent pieces of data.

How JavaScript Can Now Be Executed in Oracle Database — As of version 21c, released last month, Oracle Database can now execute JavaScript, powered by GraalVM. This post digs into the technicalities of how it works.

Why I Built Litestream, a Streaming Replication System for SQLite 3 — The creator of BoltDB, a Go powered key/value store, curiously went to SQLite for his next database project and this post explains why he thinks SQLite plus replication (as offered by Litestream) is the future for many use cases.

How to Trigger Cloud Run Actions on BigQuery Events — Using serverless functions for HTTP services is great, but some of the most interesting and seamless use cases come when functions are run automatically in reaction to other events. Here, we see how you can create an elaborate sort of database trigger in BigQuery using event triggers and a containerized Cloud Run function.

Babelfish: The Elephant in The Room? — Last year, AWS announced Babelfish, a project to provide a SQL Server compatible endpoint for PostgreSQL and that will be open sourced this year. Will an open source version easily integrate with Postgres though? Not exactly, and that’s the problem examined here.

Machine Learning Offers Fresh Approach to Tackling SQL Injection Vulnerabilities — How academics are experimenting with using reinforcement learning to automate the SQL injection exploitation process.

Guide for Linux Users: Tune PostgreSQL & EDB Postgres Advanced Server — Download this guide to get a set of recommendations to configure and tune PostgreSQL and EDB Postgres Advanced Server.

Migrating a Commercial Database to Open Source with AWS SCT and AWS DMS — AWS’s push to get people off of commercial database engines and on to open source databases (hosted at AWS, naturally) continues. Here, we learn about the AWS Schema Conversion Tool and the Database Migration Service which can assist such migrations.

'10 Things I Hate About SQL' — The author is quick to note that he thinks SQL is awesome, but that doesn’t mean there aren’t things about it he finds less than awesome. He’s wrong about point two, but if you’re in the mood for an opinion piece.. 😄

▶  Answering an Advanced SQL Question from Amazon — If you want to stretch and test your SQL knowledge, this guy’s videos seem pretty useful generally. This one tackles an advanced SQL challenge raised by Amazon in a data scientist interview.

TimescaleDB 2.0 Now Generally Available — The Timescale folks revealed TimescaleDB 2.0 late last year but now it’s GA. If you want an open source, highly scalable, time series database sat atop Postgres, check it out.

Comparing Fauna and DynamoDB: Features & Pricing — Fauna and DynamoDB are both serverless databases, but their design goals, architecture, and use cases are very different. This post discusses where they shine, where they don’t, and how they offer fundamentally different value propositions.

Use IBM Cloud SQL Query to Analyze VPC Network Traffic from IBM Cloud Flow Logs for VPC — A bit niche and IBM-specific but it’s neat to see SQL for analyzing networking logs.