Curious (Clojure) Programmer: Weekly Bits 02/2022 - Domain Modeling, Java DNS caching, RDS upgrades with Terraform

Clojure

A few Clojure-related links:

Donut.system: your new favourite component library? (by Daniel Higginbotham)
The new Clojure "iteration" function
Is there any performance penalty to using vars instead of function references?
only matters when they’re on a hot path or very fast. If they take microseconds or more to execute I wouldn’t be bothered by it
— Ben Sless

PurelyFunctional.tv newsletter - domain modeling series

This is not really Clojure-specific, but I started reading through a bunch of posts about domain modeling by Eric Normand (a big FP & Clojure advocate). It starts with PF.tv newsletter # 446 and he’s still publishing new posts.

There’s a lot of great content and insights. My favorite one, so far, has been 447: Domain model fit where he talks about reference back to reality and, as an example, misuse of the Decorator pattern. It also links to two former episodes talking about the Decorator pattern in more detail:

nippy serialization issues with Joda DateTime

We store user session data in Redis via nippy.

After upgrading the library to a newer version, we found a serialization issue in one particular use case:

clojure.lang.ExceptionInfo: Cannot thaw object: `taoensso.nippy/*thaw-serializable-allowlist*` check failed.
This is a security feature.
See `*thaw-serializable-allowlist*` docstring or https://github.com/ptaoussanis/nippy/issues/130 for details!
class-name: "org.joda.time.DateTime"

The solution was to, as they suggest, whitelist all the classes in the org.joda.time package:

  ;; saving joda DateTime instances in the session requires custom allow list for nippy
  ;; see https://github.com/ptaoussanis/nippy/issues/130
  (alter-var-root #'taoensso.nippy/*thaw-serializable-allowlist*
                  (fn [allowlist] (conj allowlist "org.joda.time.*")))

Notice that the default allowlist contains, for example, java.time classes:

*
taoensso.nippy/*thaw-serializable-allowlist*
#{"java.lang.NullPointerException"
...
  "java.time.LocalTime"

See the nippy issue for more details.

Java / JVM

Regular expressions - character classes

In the Inside Java podcast, I learned about https://dev.java/, a nice resource for java developers.

One of the resources is about regular expressions and, in particular, character classes. I was well aware of the stuff like [a-zA-Z], but the usage of interesections via && was new to me.

E.g., this means "a through z, and not m through p: [a-lq-z] (subtraction)".

[a-z&&[^m-p]]

Java and DNS caching

In Some ways DNS can break, Julia Evans talks about various ways DNS might cause you headaches.

One of them is "Java caching DNS records forever". This only applies to situations when SecurityManager is enabled (not mentioned in the article) or when you explicitly tell Java to do so.

However, it made me explore java DNS caching in more detail. I found a great article Host name resolution in Java which dives deep into the topic.

Below, I summarize a few important things I have learned.

If you don’t customize it, the default values for caching / TTLS in Java are:

30 seconds for successful results
10 seconds for negative caching
- Note that javadoc says, that the default for `networkaddress.cache.ttls is to cache forever if a security manager is enabled and implementation-specific otherwise. The OpenJDK implementation I linked uses the default 30 seconds.

`networkaddress.cache.ttl` and `networkaddress.cache.negative.ttl` are java security properties:

That means you cannot set them with -D like this:

# this doesn't work!
java -Dnetworkaddress.cache.ttl=5 ...

Instead, you have to set it in the java.security policy file. Notice also how the default for the negative cache ttl is set in the policy file, not in the source code as with the "positive" cache ttl:

networkaddress.cache.ttl=5

# this is the default value - if you delete this line,
# the default value set in the JDK code is 0
networkaddress.cache.negative.ttl=10

You also cannot read such properties via System.getProperty. Instead, you have to read them via java.security.Security#getProperty: https://github.com/jumarko/clojure-experiments/blob/master/src/clojure_experiments/networking/dns.clj#L23-L33

(System/getProperty "networkaddress.cache.negative.ttl")
;; => nil
(System/getProperty "sun.net.inetaddr.negative.ttl")
;; => nil

(java.security.Security/getProperty "networkaddress.cache.negative.ttl")
;; => "10"
(java.security.Security/getProperty "networkaddress.cache.ttl")
;; => nil

Java calls native function `getaddrinfo` (on Linux)

Adn this apparently doesn’t work with round robin DNS load balancing technique because getaddrinfo sorts the IP responses it receives.

AWS & Cloud

RDS upgrade through terraform - replaced instance, missing tables/data

I recently attempted a major upgrade of PostgreSQL engine version for an AWS RDS instance This was an update from 9.6.22 to 13.3.

Preparation

I read through the AWS document and made a snapshot of the database before upgrading. It turned out this was a really good idea.

The instance is managed via Terraform. After a few failed attempts, I finally convinced Terraform to perform the upgrade. But to my great surprise, when the instance came back all the data was missing including all the tables!

Prerequisites

To do the major upgrade I had to:

add allow_major_version_upgrade = true. If you don’t do that you’ll see this error

│ Error: Error modifying DB Instance abc-xyz: InvalidParameterCombination: The AllowMajorVersionUpgrade flag must be present when upgrading to a new major version.

Update the instance class - we were using old-school db.t2.small so I had to upgrade to db.t3.small first Otherwise it reported this error:

│ Error: Error modifying DB Instance abc-xyz: InvalidParameterCombination:
RDS does not support creating a DB instance with the following combination: DBInstanceClass=db.t2.small, Engine=postgres, EngineVersion=13.3, LicenseModel=postgresql-license.
For supported combinations of instance class and database engine version, see the documentation.

The surprise

After performing the upgrade I checked the app using the database and found a bunch of ERRORs like this:

Caused by: org.postgresql.util.PSQLException: ERROR: relation "xyz" does not exist

Well, that was unexpected!

After talking to a colleague and searching on the Internet, we found that, for major upgrades, terraform simply deletes the instance and creates a new one:

If you had simply attempted to update your main.tf file with a new database version, Terraform would have deleted the existing database and created a new database with the newer version. You would have needed to recover your data from a backup.

Takeaways

Always, always make a backup/snapshot before attempting any significant RDS upgrades
Study the output of terraform apply carefully

About (2): Had I studied the terraform apply output carefully, I would have noticed something strange: terraform trying to recreate the instance instead of updating it in place.

Cloudonaut - Comparing API Gateways on AWS

A very useful resource from the Cloudonaut guys about 5 different types of API gateways offered by AWS.

It’s a bit older but quite useful: https://cloudonaut.io/comparing-api-gateways-on-aws/

The 5 types include:

API Gateway REST API - most mature, supports user/tenant based throttling
API Gateway HTTP API
API Gateway WebSocket API
API Gateway AppSync (GraphQL)
Application Load Balancer - cost effective and very simple to use but not able to transform requests/responses

Books

Api Security in Action

This is a wonderful book I briefly mentioned in the previous weekly summary

Here’s an ugly mindmap draft I created for chapter 4 (Session cookies):

Api Security in Action - Mindmap for Chapter 4

Practical Monitoring

Practical Monitoring is a wonderful little book teaching principles and practices of good monitoring.

I’ll talk more about this in some of the future blog posts.

50 Quick Ideas to improve your user stories

A short book from Gojko Adzic.

It’s structured as 50 somewhat independent tips & ideas to improve user stories writing, planning, etc. I think it can be immensely useful if you are involved in (software) product development.

MISC

Software bill of materials

This is an interesting area of focus in the security and open source community. I recently stumbled upon this topic when listening to the podcast with David A. Wheeler: Open Source Security with Dr. David A. Wheeler

We also got a question about it from one of our CodeScene customers.

Here’s a useful resource explaning the idea: https://fossa.com/blog/software-bill-of-materials-formats-use-cases-tools/

A software bill of materials is an inventory of all software components (proprietary and open source), open source licenses, and dependencies in a given product.
A software bill of materials (SBOM) provides visibility into the software supply chain and any license compliance, security, and quality risks that may exist.

Weekly Bits 02/2022 - Domain Modeling, Java DNS caching, RDS upgrades with Terraform