In the latest release of Leapfrog Geo v 5.1.0 important changes have been made to the imported points object. While on the surface you may not notice much difference, we expect that you will notice a great improvement in performance.

The biggest change is to the underlying storage structure – points have been moved out of SQL Lite and into a new binary object. As a consequence, importing, reloading and appending points is much, much faster.

An example: a 7.2 million row points file (22 numeric and 1 category column) that previously took ~50 minutes to load is now loaded in 4.5 minutes.

Video: Comparison of loading a file to existing points database vs the upgraded points database.

In addition, the change in database structure means we no longer suffer the restriction of being limited to a single process. When points get modified (e.g. edited in the table editor, or reloaded), the processing of the output columns on that table are run in parallel, not sequentially as before. Also, the processes run in the background, meaning that after you click Import, you can continue using Leapfrog Geo without limitations while the load progresses. Previously the project would have been non-responsive while certain operations were completed.

It is important to understand that while the loading and manipulating of point data is much faster, there is no change in the speed of the processing of downstream objects.

All existing points objects are automatically upgraded to the new structure when a project is updated. For projects with large points data sets this will likely result in a reduction in overall project size, as the new storage is considerably more space efficient. The upgrade process itself (migration of existing points sets filter to a new object, along with any filters and links) is quick. Upgrading of a project with 2 large points objects (12.3M and 1.5M rows respectively) with no down-stream dependencies took <60 seconds, and the overall project size decreased from 13.00 Gb to 9.70Gb.

Please note that upgrading a project will trigger reprocessing of all downstream objects dependent on points, which in some projects may take considerable time. We’d like to remind you that it is always prudent to back up projects before upgrade, and to schedule the upgrade for non-critical times.

Below are features designed to improve usability:

Fix Errors

A small but significant change has been made in the way mapped values are handled in the new points database. The options for ‘fixing errors’ remain the same (i.e. omit values, specifying replacement values, or in some cases keep) and as before, the user can make manual edits of values in the table view. But you now have better visibility of how the final values are derived. A hover-over tool tip in the table view shows the original value, the mapped or edited value (edits over-ride mapping) and the final value that will be used in all downstream processes.

Calculations and Filters on Points

The same calculations and filters engine used on block models is now also available on points.

Filters can be created directly from categorical or numeric data, without having to first create a manual category selection, saving both clicks and effort. This can be done using the familiar quick filter builder, or from within calculation tool. Any pre-existing filters are automatically upgraded.

Filters in the Calculations and Filters section.

Numeric and Category calculations on any of the imported variables can be constructed using ‘If’ statements and a range of pre-defined mathematics or basic operators.