Khepri is still under active development. It is safe to use in production. However, the API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.18.1

Several bug fixes and optimisations from Khepri 0.17.8

See the release notes of Khepri 0.17.8.

Horus 0.5.0

Horus was updated to version 0.5.0. This release makes it possible to run a cluster of Khepri members using a mix of Erlang versions. For example, some members on Erlang 27 and some members on Erlang 28.

Before this change, it was impossible to do so because a transaction, a projection or a store procedure created by a member running Erlang 28 would fail to run on the Erlang-27-based member. That member would then be unable to apply anything and follow the cluster. This could even lead to a loss of quorum until all members were running Erlang 28.

See #418.

Other changes

• Fix type spec of khepri_machine:api_behaviour_to_machine_version/1 (#387).
• Fix the doc of machine version 3 (#391).
• Update Ra from 3.1.2 to 3.1.10 (#383, #419).

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.18.1

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.18.1"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.18.1

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.

Khepri is still under active development. It is safe to use in production. However, the API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.17.8

Optimised update and restore of projections

Khepri is faster and consumes less memory when it needs to update or restore projections. Testing shows a 25% drop in memory consumption while updating or restoring projections, while taking 50% less time to update them.

See #415.

Horus 0.5.0

Horus was updated to version 0.5.0. This release makes it possible to run a cluster of Khepri members using a mix of Erlang versions. For example, some members on Erlang 27 and some members on Erlang 28.

Before this change, it was impossible to do so because a transaction, a projection or a stored procedure created by a member running Erlang 28 would fail to run on the Erlang-27-based member. That member would then be unable to apply anything and follow the cluster. This could even lead to a loss of quorum until all members were running Erlang 28.

See #418.

Other changes

• Replace catch by try/catch blocks (#385).
• Pass machine_version when applying init-time commands (#388).
• Fix an issue with the prop_state_machine property-based testsuite (#389).
• Fix a bug with the handling of the #if_not{} condition (#394).
• Fix a bug when a trigger is evaluated against just-deleted node (#405, #398).
• Fix a bug in the generic tree walk function when a new node is created (#412).
• Fix an invalid return value if khepri_condition:is_valid/1 (#413).
• Update Ra from 2.16.12 to 2.16.13 (#417).

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.17.8

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.17.8"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.17.8

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.

At this point, Khepri should be considered Beta. The API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.19.0-beta.2

New snapshot strategy

The snapshot strategy was replaced to avoid making snapshot under load or too often, leading to even more contention.

The new strategy is based on the amount of memory and disk space a new snapshot will free. In addition to this logic, a snapshot will never be requested before N seconds have passed since the previous snapshot.

Overall, this improves the responsiveness of the Khepri state machine under load spikes.

See #392.

Improved cluster-wide compatibility

When a cluster is using several different versions of Erlang/OTP, it could happen that a transaction or a projection would fail on the nodes using an older version of Erlang. This was caused by the fact a cluster member running a newer version of Erlang could generate a Horus standalone function that could not be loaded by a member running an older version, because the underlying Erlang module would not be compatible.

With the update to Horus 0.5.0, a member running e.g. Erlang/OTP 28 will generate a standalone function that should be compatible with another member running Erlang/OTP 27.

See #418.

Other changes

• Add an option to override the ETS table protection setting of a projection (#393).
• Optimise a tree walk when a component pattern matches a specific list of nodes (#395).
• Optimise keep_while cascading deletes (#406).
• Optimise path pattern matching when #if_all{} and #if_any{} are used (#407).
• Refactor several parts of the code to make new features easier to implement and review (#396, #403, #404, #408, #409, #411).
• Ra was bumped from 3.1.2 to 3.1.10 (#383, #419).
• Replace a hard-coded value in a testcase (#402).
• Fix function specs (#387).
• Improve documentation (#391, #399).

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.19.0-beta.2

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.19.0-beta.2"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.19.0-beta.2

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.

At this point, Khepri should be considered Beta. The API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.19.0-beta.1

New snapshot strategy

The snapshot strategy was replaced to avoid making snapshot under load or too often, leading to even more contention.

The new strategy is based on the amount of memory and disk space a new snapshot will free. In addition to this logic, a snapshot will never be requested before N seconds have passed since the previous snapshot.

Overall, this improves the responsiveness of the Khepri state machine under load spikes.

See #392.

Other changes

• Add an option to override the ETS table protection setting of a projection (#393).
• Optimise a tree walk when a component pattern matches a specific list of nodes (#395).
• Optimise keep_while cascading deletes (#406).
• Optimise path pattern matching when #if_all{} and #if_any{} are used (#407).
• Refactor several parts of the code to make new features easier to implement and review (#396, #403, #404, #408, #409, #411).
• Ra was bumped from 3.1.2 to 3.1.5 (#383).
• Replace a hard-coded value in a testcase (#402).
• Fix function specs (#387).
• Improve documentation (#391, #399).

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.19.0-beta.1

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.19.0-beta.1"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.19.0-beta.1

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.

At this point, Khepri should be considered Beta. The API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.18.0

Support multi-table projections

Khepri 0.18.0 supports extended projections that need to manage several ETS tables. This is useful to maintain multiple views of the same set of records, or to organise a complex structured view of that set of records accross several tables to provide more efficient updates and lookups.

To declare a multi-table projection, you can provide the list of tables along with their specific ETS options:

Options = #{tables => #{my_table_1 => #{type => set},
                        my_table_2 => #{type => bag}},
            read_concurrency => true},
khepri_projection:new(my_projection, fun extended_fun/4, Options).

Your projection function is then called with a map of table names/TIDs instead of a single TID:

extended_fun(Tables, Path, OldProps, NewProps) ->
    #{my_table_1 := Tid1,
      my_table_2 := Tid2} = Tables,
    ...

See #366.

Other changes

• Add an internal API to wait for a machine version or behaviour to be effective in a cluster (#374).
• Ra was bumped from Ra 2.16.12 to 3.1.2 (#340, #347, #364, #375, #378, #382).

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.18.0

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.18.0"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.18.0

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.

At this point, Khepri should be considered Beta. The API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.17.7

The release uses Horus 0.4.0. This new version brings a lock-related bug fix as well as some preparation work for future changes (to make this version compatible with future versions of Horus).

It is a new minor version but it is fully compatible with the Horus 0.3.x series.

See #379 and #380.

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.17.7

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.17.7"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.17.7

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.

At this point, Khepri should be considered Beta. The API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.17.6

The release comes with several non-critical bug fixes:

• Fix use of patterns when we want to handle several expired nodes in a single pass in remove_expired_nodes1/2 (#376).
• Handle exception from ra:trigger_election/2 when we expand a Khepri cluster; it was already the case when starting a Ra server (#369).
• Cache effective machine version after recovery (#373).
• Allow the use of ?LOG_*() macros in transactions (#377).

The remaining changes are related to docs, Dialyzer fixes or small no-op refactors:

• Refactor the code that needs to know the Ra leader when khepri_machine wants to remember if the latest command was local or remote (#361).
• Fix type specs (#363, #368).
• Use the correct guard to check if an argument is a store ID (#370).
• Fix typo if "maple" (#365).

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.17.6

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.17.6"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.17.6

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.

At this point, Khepri should be considered Beta. The API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.17.5

A couple bug fixes:

• Send the #dedup_ack{} to the leader instead of the local Ra server if the leader is known (#355).
• Fix handling of deletions when triggering stored proceudres (#357).

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.17.5

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.17.5"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.17.5

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.

At this point, Khepri should be considered Beta. The API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.17.4

Improved keep_while conditions

The focus of this release was put on keep_while conditions and in particular, bugs and performance issues. These were discovered while working on topic exchanges in RabbitMQ.

The first bug is about child nodes of a deleted parent nodes missing from the returned result. The second one was leaked entries in the keep_while reverse index in the machine state. Both were addressed by #352.

The performance issues were related to the way we squash version bumps after the handling of keep_while conditions, and the way we handled many tree nodes expiring as part of the application of one machine command. See #353.

Other changes

Horus was updated from 0.3.1 to 0.3.2 (#354).

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.17.4

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.17.4"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.17.4

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.

At this point, Khepri should be considered Beta. The API will likely evolve and future releases will likely introduce breaking changes.

What's new in Khepri 0.17.3

Faster queries

When khepri_machine computes the list of default options, it verifies which machine version is currently effective, regardless of what it supports locally, because the overall cluster might use an older machine code.

This machine version check uses Ra counters, but they remain expensive to read. This is specially visible when querying a Khepri store. A significant part of the time it takes to process a query is spent in reading the Ra counters.

This version caches the effective machine version when the Khepri machine is started and when the machine version is upgraded using a persistent_term. This is way more efficient and improves the performance of queries. This improves the life of the machine itself, but the impact is less visible.

See #349.

Other changes

• Handle {error, normal} return value from Ra in khepri_machine:process_sync_command/3 (#350).
• Ignore keep_while condition at time a tree node is created when the keep_while targets a child node (#351).
• Address several warnings from Dialyzer (#345, #346).

Download

Khepri is available from Hex.pm: https://hex.pm/packages/khepri/0.17.3

Upgrade

Using Rebar:

1. Update your rebar.config:

   %% In rebar.config
   {deps, [{khepri, "0.17.3"}]}.

2. Run rebar3 upgrade khepri.

Using Erlang.mk:

1. Update your Makefile:

   %% In your Makefile
   dep_khepri = hex 0.17.3

2. Remove the deps/khepri directory. The new version will be fetched the next time you build your project.

Documentation

The documentation for Khepri is available on Hex.pm.