Bookmarks (1147)

'Everything You Know About MongoDB is Wrong' — Both fairly and unfairly, MongoDB has been the target of numerous assertions over the years that they’re keen to correct and refine. Does MongoDB really lose your data? Is it ‘easy’? Find out more here.

Pydis: A Redis Clone in Python 3 (Sort Of) — The developer’s goal here was to dispel myths about performance when using ‘interpreted’ languages like Python, but I think it proves more of a point about how such languages make it easy to glue things together (here, it’s libuv and hiredis) for an acceptable result.

Postgres 13.1, 12.5, 11.10, 10.15, 9.6.20, and 9.5.24 Released — Updates to all of the currently maintained lines of Postgres to tackle three security vulnerabilities, one of which unfortunately lets certain types of trusted user execute arbitrary SQL as a superuser.

K8ssandra: An Apache Cassandra Distribution for Kubernetes — Includes automation for operational tasks such as repairs, backups, and monitoring.

7777: Your Remote AWS Database on Local Port 7777 — This is a neat idea. It’s a paid piece of software, but it automates the job of creating AWS jump servers and SSH tunnels to RDS servers and works with MySQL, Postgres, MariaDB, and Aurora.

DynamoDB Design Patterns for Single Table Design — I hope you’ve got a lot of time because this goes into serious depth.

Working with Astronomical Data in Python — Early days but this is the first public draft of a book being written about using SQL, Astropy, Pandas and Matplotlib to work with astronomical data from Python.

Writing a Postgres Foreign Data Wrapper for Clickhouse in Go — Foreign data wrappers (FDWs) provide a way for Postgres servers to interact with external services (including other database systems), such as the ClickHouse OLAP DBMS covered here.

🤡  A Twitter DM-Based Database for Twitterbots on AWS Lambda — Naughty? Genius? A bit of both? This approach made me smile at least. Are you going to use this for anything serious? Probably not.

Ask HN: Am I Too Late for The “Data Science” Wave? — A busy Hacker News discussion on the top of data science and whether it’s still worth getting into. In short, yes - it’s still early days.

Database Horror Stories [Video] — Tammy Bryant, Principal SRE at Gremlin, presents "Database Horror Stories" at the first Prisma Online Meetup.

Azure Cosmos DB Serverless Now in Preview — It’s in preview but Azure Cosmos DB now has a consumption-based serverless offering.

Importing 100M+ Records into DynamoDB in Under 30 Minutes — Last week we mentioned that DynamoDB now has an ‘export to S3’ feature, but what about loading data into DynamoDB at scale? Enter a batch of simultaneously running AWS Lambda functions!

Cloudflare Unveils Free Tier of Its Workers KV Store — Workers KV is Cloudflare’s key value store for its serverless edge platform and it now gains a free tier (1GB total storage, 100k reads, 1k write/list/delete ops per day) opening up to everyone using Workers. The max value size also goes from 10MB to 25MB.

Amazon Timestream vs DynamoDB for Timeseries Data — A comparison of Amazon Timestream with DynamoDB for time series data storage. There’s quite a bit of nuance involved, but part of the conclusion is that “DynamoDB is faster for targeted queries, whereas Timestream is better for analytics that include large amounts of data.”

'Why I Left IBM to Work on CockroachDB' — Despite the title, this is more an interesting story of what a self described ‘database nerd’ got up to at IBM working on its autonomic computing team and on their Db2 database products.

Elasticsearch 7.10.0 Released — The latest version of the popular search-focused database features ‘searchable snapshots’, a beta feature for directly searching snapshots (basically Elasticsearch backups!) without restoring them in full — which opens up some neat opportunities like keeping larger searchables in snapshots on S3 or Azure Storage, say.

Announcing CockroachDB 20.2 — CockroachDB 20.2 includes spatial data types and indexing, improved performance on TPC-C and TPC-H benchmarks, and introduces CockroachDB on Kubernetes.

AWS Glue DataBrew: A Visual Data Preparation Tool — Good analytics and reports rely upon good, clean data. Glue DataBrew is a new tool for helping you clean and normalize data stored in S3, Redshift, RDS, and other places which you can then use in systems like SageMaker, Redshift or Athena. Despite the rather dodgy name, this looks like a neat service and I’m...

SQLite Status Report 2020: A Slidedeck — A slide deck covering the state of SQLite which is now 20 years old. The corresponding talk isn’t here but the slides are packed with info and examples nonetheless on areas like recursive CTEs, CLI improvements, and security considerations.

Postgres Build 2020 - PostgreSQL Online Conference [Free] — Listen to 25+ talks on the latest in PostgreSQL and on how PostgreSQL helps businesses. This conference will also provide wellness sessions to promote health and revitalize. Join us from December 8 - 9. Register now.

You Can Now Export DynamoDB Table Data Directly to S3 — Rigging up a system based on Data Pipeline, EMR, or Glue and DynamoDB Streams to shuttle DynamoDB data to S3 has long been a common task, but now you can export DynamoDB table data straight to S3 in JSON or Ion format (and then query it with Athena using standard SQL).

Databricks Launches SQL Analytics — You have a data warehouse, a data lake, and now Databricks is moving into the ‘lake house’ where data processing and analytics are combined and SQL analytics aims to let you run SQL-driven BI and reporting workloads directly upon data lakes.

Amazon DocumentDB Introduces Transactions and MongoDB 4.0 Compatibility — AWS shows no sign of slowing down with any of its developments and their controversial (in some quarters) MongoDB-compatible document store is catching up with MongoDB 4.0 in the area of ACID transactions and cluster level change streams.

Migrating Large Heroku Postgres Instances to AWS Aurora Without Downtime — Heroku is often a great place to start but the flexibility of RDS is appealing particularly if there are other AWS bits and pieces you want to work with (or to save costs, as in this case). There’s a lot to process and enjoy here as there were a lot of moving parts.

Monitoring MySQL Database Performance with New Relic — Don’t take MySQL for granted. It's more complex than you may think. See key metrics to monitor in production.

Powerful New Features in Azure Cache for Redis — Back in May, Microsoft partnered with Redis Labs to bring Redis Enterprise to Azure (in the shape of Azure Cache for Redis) and they’re now bringing extra features to it.

Minimizing Read-Write MySQL Downtime at Yelp — Yelp is a significant user of MySQL and has brought together various tools to improve failure detection and execute automated recovery processes.

Why Databases Are Needed on the Edge — The proliferation of smart devices and the IoT are putting pressure on developers to adopt more sophisticated processes to manage data.

gron: Make JSON Greppable — A tool written in Go that transforms JSON into more easily greppable assignments, so you can use grep and see the context/path of the result. Useful.

NoisePage: A 'Self Driving' Database Management System — We’ve touched on this topic of self-optimizing databases a few times over the years and I think we’re going to see far more of it this decade. NoisePage is an open source, relational database (that uses a Postgres-compatible wire protocol) out of CMU that's entirely focused on testing out autonomous deployment and machine learning optimization techniques.

sql.js: SQLite Compiled to JavaScript — No persistence as it works with a virtual database file stored in memory (though it can be exported). Potentially useful for building SQL tutorials or even client side data manipulation systems. Demo here.

▶  What's So Exciting About PostgreSQL? — PostgreSQL guru Craig Kerstiens went onto the popular Changelog podcast to talk about his favorite relational database. There’s also a well produced transcript to read, if you prefer.

Election Night Prediction Modeling using PL/R in Postgres — Rather than constantly refreshing Twitter for election news, Joe Conway was busy trying to run some R-based election forecasting code in Postgres.

Procella: YouTube’s Analytical Column Store — This post is from June but we missed it at the time. It digs into an SQL-based processing engine YouTube uses that’s built on top of all sorts of Google bits and pieces. Mark Litwintschik also wrote about Procella last year.

Sir: A Local Filesystem 'git-Diffable' JSON Database — Principally aimed at Node users needing a simple data store. In a similar local, lightweight vein as Python’s TinyDB, JSDB or the Mongo-inspired minim-json-db.

Developers Increasingly Pair MongoDB with PostgreSQL, Survey Finds — An interesting finding that comes out of a new Studio3T survey of 18,000 developers. Redis is also a common pairing with MongoDB.

Results of Percona's 2020 Database Management Survey — The majority are using open source database monitoring tools, downtime and performance are people’s biggest concerns, AWS leads the way in cloud databases (though Azure is close at larger companies), multi-cloud setups are uncommon, and databases are only continuing to get larger. MySQL dominates amongst database systems here, but that’s perhaps no surprise given Percona’s origins.

DuckDB: The Little OLAP Database That Could — We’ve featured DuckDB a few times. It’s a local, in-process OLAP database system that seems to be getting a lot of love from different directions lately. Here, the author takes it for a spin and shares some TPC-DS benchmark results before calling it a ‘hummingbird of databases - small and fast’.

TimescaleDB 2.0: A Petabyte-Scale Relational Database for Time-Series — They really go all out to sell TimescaleDB here, but they have reason to be proud of their Postgres-based database which seems to go from strength to strength. v2.0 brings multi-node support, improved licensing (everything’s free), and more (as explained in these tweets).

CYBERTEC Migrator - The Easiest Way from Oracle to Postgres — CYBERTEC Migrator is an easy and user-friendly tool with a visual interface that helps you to efficiently migrate multiple Oracle databases to PostgreSQL. Newest features are close-to-zero downtime migration (CDC) or automatic code rewrite.

▶  The Most Popular Databases: From 2006 to 2020 — A bar chart that shifts and changes over time based on the popularity of different database systems (as determined by the TOPDB index). Not definitive by any means but interesting to see the shifts nonetheless.

100,000+ Books in Plain Text Format — This is targeted at people with machine learning use cases but could be ideal as a dataset for all sorts of things. It’s a 37GB download though. There’s also a 100GB file containing source code for similar purposes. It’s all explained in more detail here.

Memgraph 1.2 Release - Implementing The Bolt Protocol v4 — “We thought that it would be interesting to discuss the Bolt Protocol for those of you who are interested in graph databases, and explore what goes into being wire-compatible with another database system.”

What Is This Mythical Zero Downtime Database Migration to the Cloud? — Join us for a CIO led panel discussion on the issues and risks and the technology available to streamline the experience.

How Dolt Implemented Pushdown Filters to Make Queries Faster — ‘Pushdown’ is where you move the predicates in a WHERE clause to be under the join, reducing the number of rows you have to examine. Dolt itself is a SQL database that adopts many of Git’s concepts to provide version control primitives over data.

AWS Nitro Enclaves: Isolated EC2 Environments to Process Confidential Data — Aimed at AWS users processing ‘highly sensitive data’, Nitro Enclaves lets EC2 users create isolated environments on instances powered by Nitro.

▶  How a Man Learning Postgres Exposed 40K Voters Purged By Mistake — A long-time database administrator was running queries on voter data to learn how to use Postgres and stumbled over a significant voter purge error. May you too be able to use your powers of database good to improve democracy :-)

Microsoft Overhauls Excel with Live Custom Data Types — A few months ago we mentioned scientists were renaming human genes due to auto-formatting issues in Excel(!) but now Excel is getting more powerful with support for custom live data types.