Bookmarks (1038)

Introducing Pebble: A RocksDB Inspired Key-Value Store Written in Go — The folks at CockroachDB created Pebble as a RocksDB-compatible replacement for their storage engine. This post explains the why and the how of their process.

Devops Engineer (Full time / Full Remote) — We're looking for a devops to join our fast-growing, well-funded company that makes thousand of African fashion designers happy.

Apache Tika: A Content Analysis Toolkit — The Apache Foundation incubates a lot of great software, so much in fact that much of it can fly under the radar, but this is pretty neat if you need to parse and extract metadata and text from almost anything.

TerminusHub: Distributed Revision Control for Structured Data — From the folks behind TerminusDB, an open source in-memory knowledge graph database, comes TerminusHub, a place to collaborate and share databases while keeping them synced. (Think how GitHub allows git users to collaborate..)

A Database for Every REPL on Repl.it — Repl.it is an online tool for writing (and even running) code in an in-browser IDE. Now, they’ve added their own key/value store into the mix, usable from each Python, Node or Go REPL.

Index Improvements Coming to Postgres 13 — We’ve linked to numerous articles over the past few months about how indexes will be improved in Postgres 13 but this rounds up six of the biggest benefits in one place complete with code examples.

My Journey Making QuestDB — The story of how QuestDB, an open source database for time series, events, and analytical workloads, came to be.

Old, Good (Relational) Database Design — It’s back to basics for Elnur, thinking about what a robust and well-thought out database design really is, and it comes down to normalization, data types, constraints, and indexes.

PostgreSQL 13 RC 1 Released — Postgres 13 is shaping up to be a huge release for the popular, open source relational database. The beta release notes cover the essentials.

JavaScript for Data Science: A Book — A book available both commercially and to read online aimed not specifically at JavaScript developers (as it provides a neat introduction to JS in its own right) but at anyone looking to use JavaScript as a bit of a ‘Swiss Army knife’ for working with datasets. This is one of those great resources you’re likely to pass on to someone who really needs it.

Oracle Touts an Exadata Backlog — Oracle reported modest revenue growth in its latest quarter and said that moving more of its flagship database software to the cloud remained a priority. It also noted strong demand for its Exadata platform.

🖥 Webinar: The Architecture of a Distributed SQL Database — Join us on September 23 for an under-the-hood look at the internals of CockroachDB. We’ll explore how we guarantee serializable isolation, our implementation of Raft, and discuss why we built a new storage engine to replace RocksDB.

How (and Why) GitLab Upgraded from Postgres 9.6 to 11 — Some of the GitLab team explain the precise maintenance process they took to execute a major version upgrade of Postgres on their systems. And, even better, they recorded the whole 2 hour process so you can see how it was actually conducted! It doesn’t make for exciting viewing, but it’s a very uncommon look behind the database ops curtain at a major company.

Generating a Normal Distribution in SQL — Postgres’s tablefunc extension provides a variety of functions that return tables, including sets of normally distributed random values.

The CMU Database of Databases — An online encyclopedia of 714 (and counting) database systems, thanks to Carnegie Mellon University. We might have to spend some time going through this! It includes a leaderboard showing the most common countries of origin (US is #1 unsurprisingly), programming language used (Java at #1), and more.

10 Exciting Big Data Statistics — Some fun stats that Datanami has pulled from a variety of places including that the ‘global datasphere’ will be 175 zettabytes (175 billion terabytes) in size by 2025 and that more data is created every hour in 2020 than in the entire of the year 2000. No wonder database technology remains a hot sector!

Get 3x Higher Azure MySQL Throughput & Reduce Latency by 66% — See how the multi-cloud DBaaS provider ScaleGrid fares in this Azure MySQL Performance Benchmark vs. Azure Database for MySQL. Which provider offers the best throughput and latency performance for different workloads?

SQL Fiddle: A Tool for Online Testing of Database Queries — This isn’t new but as we haven’t linked it before and we use it quite a bit to test quick SQL queries and syntax, here you go. Set up your schema (which can be empty) on the left then issue queries on the right. Supports specific (slightly old) versions of MySQL, Oracle, Postgres, and SQL Server. DB Fiddle is a similar tool covering newer versions of MySQL and Postgres.

Hasura Raises $25 Million Series B; Adds MySQL Support to Its GraphQL Service — Hasura provides a service and tooling for adding GraphQL API support to existing databases and has extended support to MySQL in addition to Postgres.

Cassandra Gets an Indexing Upgrade — A new data storage framework for indexing on Apache Cassandra released this week by DataStax is aimed at easing adoption by eliminating tradeoffs between scale, stability and performance.

AWS Announces Data API for Amazon Redshift — Amazon Redshift can now be accessed using the built-in Data API which simplifies access by removing the need to manage database connections and credentials.

Using PostgreSQL to Offload Real-Time Reporting and Analytics from MongoDB — Weighs up the advantages and disadvantages of moving read-heavy analytics off a primary MongoDB database to PostgreSQL.

Kodda: A Better Way to Integrate Your Data — Moving and modeling data should be easy, Kodda harnesses the power of SQL to automate the creation and management of data pipelines allowing your teams to focus on delivering insights.

Mining for Logic Bugs in the Citus Extension to Postgres with SQLancer — One of those things you’re unlikely to need to do, but it’s nice to know how such problems are approached. SQLancer is a tool we’ve linked to before that helps you detect logic-related bugs in database systems.

Why BIGO Chose an HTAP Database over MySQL for Horizontal Scaling and Complex Queries — The creators of TiDB (an open-source, MySQL-compatible, horizontally scalable database) seem to be on a roll with these case studies, but unlike many business-oriented case studies they touch on a few interesting technical details.

The Life of a Data Byte — A byte is just eight bits.. right? Not always! Jessie takes a pretty interesting ‘walk down memory lane’ (ha!) to look at the history of bits and bytes from 1951 to the current time. You’ll learn at least an interesting factoid or two here.

Full-Text Search Battle: Postgres vs Elasticsearch — Less a battle and more a basic comparison of techniques with a simple benchmark, but may be of interest.

ArangoDB 3.7: A Big Step Forward for Multi-Model — It’s been fun watching ArangoDB, a multi-model database that supports multiple NoSQL models out of the box, develop over the past several years. With version 3.7, Arango claims to be the “most flexible and scalable graph database available” which is quite the claim given the competition. There are new fuzzy search capabilities available and HTTP/2 support too.

Oracle Introduces MySQL Database Service — Oracle is now offering a hosted and managed MySQL service running on Oracle Cloud and boasts that it’s the only service 100% developed and supported by the actual MySQL team.

▶  Discussing Document Databases and Mongo with Joe Karlsson — An easily accessible podcast (in true Scott Hanselman style) about MongoDB and how document databases differ from classic table-based systems, with a MongoDB engineer as the guest. 32 minutes.

The Inner Workings of Oracle Development — Thoughts on a Hacker News thread I somehow missed at the time that touched on some of the processes and approaches used at Oracle to work on its database. The key to success? Millions of tests.

Heroku's Streaming Data Connectors and Best Practices — Heroku has three managed data service offerings: Heroku Postgres, Heroku Redis, and Apache Kafka on Heroku. This article looks at how Heroku’s streaming data connectors help users integrate Kafka into bigger systems.

Using Envoy Proxy’s New PostgreSQL & TCP Filters to Collect SQL Statistics — The new PostgreSQL filter from Envoy Proxy makes it easy for developers & ops engineers to collect SQL statistics.

How Facebook Accelerates SQL at Extreme Scale — Serving SQL queries on a petabyte of data is one thing, but delivering it at Facebook’s scale is something else entirely. They’ve used an open source virtual distributed file system called Alluxio to do it..

▶  Database Penetration Testing and Privilege Escalation — A 40-minute exploration of database exploitations (MariaDB, MySQL) via the use of user-defined functions.

A Look at TTL Indexes in Mongo — When someone as smart as Brandur says a MongoDB feature is *“one of the most useful new database features in years”, I listen. TTL indexes provide a way to automatically prune old data. Redis and DynamoDB have similar concepts and I use TTLs on the latter to run engineeringblogs.xyz :-)

Why PostgreSQL 13 Will Be a 'Lucky' Release — See why the author thinks that 13, often maligned as an ‘unlucky’ number, will be lucky for Postgres — it adds features like incremental sort, parallel vacuum, and improves performance of B-tree indexes.

Redis Labs Raises $100 Million, Becomes a Unicorn — The company behind the Redis database/data structure store has raised a large round of funding and taken its valuation to over $1bn.

Try CockroachDB Free for 30 Days and Get a T-Shirt 👕 — Scale fast, survive anything, deploy anywhere. CockroachDB is the SQL database for developers building modern applications. Try for free today and we'll send you a Cockroach t-shirt.

Microsoft Now Developing Its Own Hadoop — Microsoft’s Azure division is working on its own distribution of Hadoop under its ‘HDInsight’ brand.

AWS EBS Gets New Volume Type (io2) with Higher Durability and IOPS Aimed at Databases — AWS’s Elastic Block Storage service now has a new volume type aimed at ‘high-performance, business-critical databases and workloads’ (and cites SAP HANA, SQL Server and IBM DB2, in particular) which promises 100x higher durability and 10x more IOPS/GiB than other options.

How Wix is Able to Host 1 Million Sites Per MongoDB Cluster — When you need to store millions of databases with multiple collections efficiently, what do you do? This is exactly the question Wix faced when working on its Wix Data service.

How to Turn An Ordinary gzip Archive into a Database — How specially crafted gzip archives can be used as a database-like store.

MongoDB Grows Asian Footprint Through Alibaba Cloud — In an effort to expands its presence in the Asia-Pacific region MongoDB is now working with Alibaba Cloud who will offer MongoDB via its ‘ApsaraDB for MongoDB’ service.

Using askgit to Query Git Repositories with SQL — askgit is a tool (written in Go) for running SQL queries against git repos and can be used to create reports, check things, analyze codebases, etc.

immudb: A Lightweight, High-Speed Immutable Database — An open source data that boasts no data mutation APIs at all and a ‘tamper-evident’ history system. It’s built in Go. GitHub repo.

Postgraphile: Quickly Get a GraphQL API for Your Postgres Database — Work on v5 of this popular Node.js tool is underway, but meanwhile we get a 4.8 release which supports ‘enum tables’ and all of Postgres’s built-in geometric types. If you want to offer a Postgres database up over a GraphQL-based API, it’s worth checking out.