Upgraded to HTTPS

The blog has been upgraded to https. Access it using https://www.qlambda.com.
The previous URL http://www.qlambda.com redirects to the above.

Analysis of Flocking Patterns and Relations

Movement of humans and analysing the movement patterns is an interesting problem. It can be considered to flocking behaviour in species like birds, swarms. Brownian motion can be used to model such collective movements. Representing the raw data from the movement of humans across locations in tabular form or relational model is inefficient. So modelling such data on any cloud platform without a graph based model is limited. An example is BigQuery. Advantage of using BigQuery is that the Google takes care of the infrastructure and massive amount of data streams, and it is NoSQL at the storage layer, but it is still tabular in nature and a relational model is not the most apt way when it comes to modelling location data, movement patterns and making sense of the information for further use in recommendation engines and such.

Looking at various Graph databases at present, OrientDB looks very capable and performant when compared to the well known Neo4j. AllegroGraph is cool if we use RDF and SPARQL or if we use Prolog for reasoning, but it does not have support for Gremlin, which I think Prolog makes up for it, though that would have been a nice addition. The advantage of using an Apache TinkerPop-enabled data system is that we can use any backing datastore like OrientDB, Neo4j, Apache Spark, without having to use the datastore's own DSL. Gremlin graph traversal langauge is to Graph DB, what SQL is in a relational datastore and it makes working with such systems a pleasent experience instead of having to fiddle with DSLs for each different datastores one would encounter.

Inspiring are those words - Apple Special Event 2017

The sound of Steve Jobs, thought provoking, inspiring are those words, the opening of Apple Special Event 2017.

There's lots of ways to be as a person.
And some people express their deep appreciation in different ways.
But one of the ways that I believe people express their appreciation to the rest of humanity is to make something wonderful and put out there.
And you never meet the people, you never shake their hands, you never hear their story or tell yours.
But somehow in the act of making something with a great deal of care and love, something's transmitted there.
And it's a way of expressing to the rest of species our deep appreciation.
So we need to be true to who we are and remember what's really important to us.

Captivates mind in ways more than one.

Data Archival with Optical Disk to withstand EMP

Optical disk are a safe choice if we need any data to withstand an EMP (Electro Magnetic Pulse) attack. And blu-ray disks are cost effective when comparing it with magnetic storage and shields combined. Say for 1 TB external hard disk drive, current approximate price is USD 50. A 5-pack 25GB recordable blu-ray disc costs around USD 15 which is more that 1 TB storage. Even accounting for the initial price of the disc burner and software, blu-rays are cost effective as we archive more data. Now getting rest of the gear to work after an EMP attack is a different matter, however, the data is preserved.

Run LFE on Android with Termux

Termux on Android runs Erlang as described in a previous post. I tried compile, install LFE with Termux, but it failed, probably due to path prefix, after which erl starting crashing as well.

We can however pre-compile it and transfer bin, ebin and include files with the same directory structure to $HOME/bin/lfe. Then add the path to .bashrc.

export LFE_BIN=$HOME/bin/lfe/bin
export PATH=$HOME/bin:$LFE_BIN:$PATH

Set executable permission to the files in lfe/bin with chmod +x lfe*.
Now run lfe which will start the LFE REPL!



Amazing isn't it? We have a full blown Lisp along with the power of Erlang on our fingertips now.

Gorilla-REPL with boot

Gorilla-REPL is analogous to what IPython notebooks are for Python. It provides a web page with markdown and code where one can experiment on various data sets and save the results, use it for code snippets, documentation and such. And it works pretty decently. Gorilla-REPL provide lein plugin integration. Using it with boot can be done sooheon/boot-gorilla library. A sample boot task is shown below.

#!/usr/bin/env boot

(set-env!
    :source-paths #{"src" "test"}
    :resource-paths #{"resources"}
    :dependencies '[[org.clojure/clojure "1.9.0"]
                    [sooheon/boot-gorilla "0.1.1-SNAPSHOT"]])

(require '[sooheon.boot-gorilla :refer [gorilla]])

(deftask grepl []
  (gorilla :port 8002 :ip "127.0.0.1" :block true))

Start the Gorilla-REPL using boot grepl and we will see something similar as below.

Gorilla-REPL: 0.4.1-SNAPSHOT
Started nREPL server on port 52660
Running at http://127.0.0.1:8002/worksheet.html .
Ctrl+C to exit.
<< started Gorilla REPL on http://127.0.0.1:8002 >>

Navigate to http://127.0.0.1:8002/worksheet.html to see the Clojure notebook. Required dependencies can be added to the boot build script, which will be available in the worksheet. Worksheets are saved as Clojure code. We can use gorilla-repl viewer to view these file online. Sample workbook, fibonacci.clj.

Full code added to codebook repo at GitHub.

Install OneDrive on macOS High Sierra

With a clean install of macOS High Sierra, the default file format is APFS (Apple File System) for SSDs and to make it Unix like generally the choice is the one with case-sensitive, encrypted or otherwise. Since OneDrive expects a partition with case-insensitive format, we cannot setup local sync in that APFS system. And even if we create a case insenstive APFS volume, OneDrive fails to setup sync. The quick and easy way is to create a new partition which is not an APFS volume, and choose exFAT format. Use this as the directory for OneDrive sync and it will work fine.

Access non-static enum in Clojure

Let's say there is a Java class with a non-static enum field as below.

package com.example;

public class Encrypter {

    public enum KeyPlacement {
        PEER,
        INLINE
    }

    private KeyPlacement keyPlacement;

    public KeyPlacement getKeyPlacement() {
        return this.keyPlacement;
    }

    public void setKeyPlacement(final KeyPlacement newKeyPlacement) {
        this.keyPlacement = newKeyPlacement;
    }
}

Setting keyPlacement value from Clojure can be done as follows.

(ns core
  (:import [com.example Encrypter Encrypter$KeyPlacement]))

(doto (Encrypter.)
  (.setKeyPlacement Encrypter$KeyPlacement/INLINE))

When compiling the above source, inner classes, enums etc. gets generated with $ appended as Encrypter$KeyPlacement. So we can import those and access them from Clojure.