Prior to his gotocon CPH talk “Fast Data –Tools and Peopleware” I had a brief chat with Dave.

I was interested in hearing what had informed the design of his Ivy system, especially the combination of a rich HTML based GUI and a DSL edited in the browser.

This post is a overview of what I consider the key take aways from the chat and some of my subsequent reflections.

• Combining well known metaphors like a spreadsheet and boosting learnability example based queries with a DSL has been a very useful approach.

• Supporting power users is a challenge but doable in a mordern web app.

1) Firstly the app has to be designed to support that user type expects, like short cuts etc. Ivy is not just a editable field for code, but a complete IDE including debugger etc.

2) Secondly if you include a DSL don’t do it to boost your ego but because it provides actual leverage. Also don’t forget that putting a DSL out there which power users are going to trly on require that you can support it. This means that you have the physical resoources available for the long run.

• Building a web app offers the possibility of offloading computationally demanding operations to the backend, for instance generation of SVG graphs.

Full Disclosure: I am receiving a free GOTO ticket from the producers in exchange for blogging about the conference.

I just attended Edward Thomson’s talk “Deep Dive into Git”.

I find version control and especially git’s approach to version control especially interesting because because it is such a good examples of a hacker power tool that would add tremendous value if applied to other trades, like contract drafting.

Edward works on Microsoft’s reimplementation of git libgit2. Apparently the original git has certain issues when used as a library (especially in Visual Studio)

git is essentially distributed and branching is cheap and easy, which Edward demonstrated convincingly. But the higlight of the talk for me was Edward’s explanation of the dreaded “rebase” command. Edward succeeded where a others have failed misrably in explaning rebase in a way that actually makes sense – at least to me. Instead of merging rebase generates a set of diffs between the most recent ancestor of your present branch and the ancestor. Theses diffs are then applied to the branch your are rebasing to. When I realised this image from the Atlassian git tutorial actually made sense to me.

Related to git but not Edwards talk I recently discovered git auto commit mode for Emacs. I don’t think this would work for code, but I plan to try it out for the prose writing I do in Emacs.

Full Disclosure: I am receiving a free GOTO ticket from the producers in exchange for blogging about the conference.

As I wrote previuously I am going to the goto cph conference this year.

One speaker that I am definitively will go see is Dave Thomas. Besides being consistently entertaining Dave Thomas also has the experience required to reach into the great work of Computer Science past and show how it remains relevant the practical business challenges of today.

Dave’s company has a product called Ivy. It is a analytics product based on the programming language Q – a decendant of APL/K – and the KDb+ database.

Ivy’s primary usecase appears to be financial analytics, an area I know little about, but I find its approach to user interface fascinating. It shows how you can precent a contemporary web based GUI using familiar metaphors and maintain the ability to drop down to code when you need to.

Ivy has a very cool modern looking in-browser inteface with spreadsheet like overview of the massive volumes of data the system is designed to handle, a visual query builder and supports a wide range of visualizations. But it also has strong support for the Q language, including a debugger.

I hope to be able to chat with Dave about his thoughts on DSLs and GUIs during the conference and get his take on questions like “When do you want both?”, “What kind of IDE do you need if you want to support non-programmers programming?”, “Are projectional editors the future of DSLs?” etc. and report here on the conversation.

Full Disclosure: I am receiving a free GOTO ticket from the producers in exchange for blogging about the conference.

I am really looking forward to the goto cph conference this year.

One of the talks on my personal iteneary is How to train your corporation to prefer open source by Richard Lander from the Microsoft core.net team.

Microsoft is undergoing profound changes at the moment expecially in its approach to open source. One of the more prominent examples of this is the open sourcing of the .NET stack which I am sure Richard will be mentioning in his talk.

The irony of course is that Microsoft still inflicts the pain of closed source on its customers on a monumental scale – and as closed licensors go Microsoft unfortunately is at the bad end of the spectrum and showing no signs of becoming even more rigid with time. My last encounter with Microsoft’s licensing machine was less than a year ago, and for the umpteenth time numerous very reasonable requests for changes and clarifications to the license terms were summarily dismissed. It would be great to see a fresh approach to software licensing from Microsoft similar to how IBM has – although imperfectly – reinvented its terms and conditions.

There are of course many reasons to prefer open source to the extent possible. A prominent one is that you avoid the pain of license audits by vendors who missed their last quarter or who wants to strong arm you into using their latest solution that they haven’t been able to sell to customers voluntarily, better known as the Oracle approach

Full Disclosure: I am receiving a free GOTO ticket from the producers in exchange for blogging about the conference.

goto CPH is closing in rapidly.

As mentioned previously I am especially looking forward to David Nolen’s talk Putting the Dream Machine to Work.

Om is getting a lot of coverage and even if it is still very new, it is seeing some heavy duty prodcution usage, for instance at Prismatic. But Prismatic has gone even further that just using Om and released om-tools, which makes it even more straight forward to use Om. I especially like how it smoothes over some of the slightly ugly Javascript interop that Om requires out of the box.

Another talk I am looking very much forward to is Greg Young’s talk on Event Sourcing. Event Sourcing is a very interesting pattern which has a very natural fit to many common business scenarios, and I am looking forward to Greg Young giving us some examples of that. The benefits of the event sourcing pattern appears to me to overlap somewhat with the benefits of the Clojure based database Datomic which also emphazises the need for a solid model of time.

If you are interested in dabbling with Event Sourcing and have been inspired by David’s talk on Om/ClojureScript you should check out the JVM Soup post on Event Sourcing in Clojure. This post demonstrates how to use core.async Clojure/ClojureScripts CSP implentation, which David also recommends to do inter component communication when using Om, to do event sourcing in Clojure/ClojureScript.

Looks like I will be at goto CPH again this year.

The format this year is quite different, but looks very interesting nonetheless. Especially the Leading & Bleeding Edge track looks interesting. At this track I am especially looking forward to seeing David Nolen’s talk Putting the Dream Machine to Work.

David has created a ClojureScript framework on top of React called Om. Om is not just a wrapper around React. It leverages ClojureScript’s immutable datastructures to substitute costly object and array compares with reference equality checks. This results in quite significant performance improvements.

David in his tutorials encourage another ClojureScript feature core.async for intercomponent communication.

As Om is still very new there are not many presentations available but Functional UI programming with React.JS and ClojureScript is a very good introduction.

If you understand Norwegian (and maybe even if you don’t) Magnar Sveen and Christian Johansen have recorded a very entertaining series of pair programming screencast where they use ClojureScript, Quiescent (an alternative React ClojureScript library) and a TDD approach to create a Zombie themed Memory Flash Game called ZOMBIECLJ

Just attended a very nice overview talk on the Unity game development system at goto Aarhus 2013.

The tech in Unity itself is very impressive, especially the ability to deploy the same code base and assets to 10+ platforms, including iOS. But Unity has also put a lot of thought into how the product is licensed. Not only was it a very conscious decision by the founders that Unity be royalty free, but the same thoughtfulness has also been applied to the Unity Asset Store.

The Asset Store is Unity’s equivalent of the App Store, but instead of apps you can buy art (e.g. graphics, models, audio) and code assets for inclusion in your Unity project. Had Unity left the choice of license up to the creators of the submitted assets, it would have completely undermined the royalty free nature of the Unity engine, as each asset could potentially inject it’s own royalty model into a product created with Unity. Instead all assets are per definition subject to standard royalty free TOS & EULA, and as explained in the below excerpt from Unity’s Asset Store submission guidelines it is not possible to sell assets in the Asset Store that are subject to other/additional terms.

6.2 Licensing your Content

For consistency sake, all Asset Store offerings are covered by a license we have created. Please have a look at the End User License Agreement to be sure you are comfortable with it. If you are offering your content for free, you have the further option of selecting one of three free-licenses, which override the basics of our commercial license. Please do not include your own license terms in your offering.

It is very refreshing to see how licensing models can be designed to enhance the usefulness of a product and its eco-system, while at the same time serve as the underpinning of an indisputable financial success.

If you have any use-cases involving 3D graphics you really should check Unity out. And please note that Unity is not limited to games. Unity Studios have done some really cool non-game applications, including a simulation for training safety procedures at sea and a brilliant app for Velux that let’s you simulate the effects of installing Velux windows in your own home.

Richard Baeckman just did an excellent presentation on Java Flight Recorder at goto Aarhus 2013. Java Flight Recorder is an – enhanced – port of JRockit Flight Recorder to the Oracle(Sun) JVM. It ships with Java SE Development Kit 7u40 and includes a very nifty GUI called Java Mission Control that provides an excellent overvie of all the data collected in your recordings. I was very interested in seeing how useful JFR is when running Clojure code. Fortunately installation is very easy. After installing Java Platform (JDK) 7u40 I just added the following options to my project.clj file.

:jvm-opts ["-XX:+UnlockCommercialFeatures" "-XX:+FlightRecorder"]

On OS X you have to start the JMC from a terminal invoking the binary located at:

/Library/Java/JavaVirtualMachines/jdk1.7.0_40.jdk/Contents/Home/bin/jmc

This brings up the JMC GUI. If you start your Clojure app (in my case Rich’s ants.clj), your jvm(s) should be visible under the JVM Browser tab. From here you can start a recording by double-clicking the relevant JVM, right-clicking the Flight Recorder entry and choosing “Start Flight Recording…”. I have inlcuded a screenshot below of a 1 minute recording of ants.clj. As you can tell JFR collects an immense amount of information but the overhead is minimal, approx 3% according to Richard.

Please do note that the Java Flight Recorder and JMC are both Commercial Features, which I have written about previously here. This means that neither JFR and JMC may be used for production purposes without a separate payable license from Oracle. There is currently no public information regarding prices and other conditions available from Oracle.

It was suggested to me to look into the concept of Flexible Contracts, a new approach to contracting for agile it projects championed by Gabrielle Benefield who will be providing training in agile practices in connection with goto Aarhus 2013.

Flexible Contracts are born out of the observation that traditional contract models are a very poor fit for agile IT projects. This is hardly a novel observation, but Gabrielle’s work (and that of her collaborators) distinguishes itself by actually having produced a practical alternative, which they have commendably licensed under the CC Attribution 3.0 Unported license. Only the lite version – Flexlite 0.1 as it is designated – appears to be presently available, and can be accessed if you are willing to sign up to the Flexible Contracts mailing list here.

It is worth noting that the overview attached to the contract clearly states that the Flexlite contract is to be considered a “minimum viable contract”, but after having reviewed the document I still find that the current version lacking in a number of ways.

Below I will outline a subset of the issues I have identified and for some suggests some possible remedies. Please note that I am commenting on the UK version (which you might want to keep handy if you want to make sense of my comments) of the contract which is the system of law I am most familiar with, but most of my comments should apply to both versions, with the exception of the issue of “best endeavours” where there is a significant difference between US and UK law.

Statetment of Target Outcomes (SOTO)

The contract focuses on the use of SOTOs instead of specifying the desired results through requirements or a more traditional solution specification. A SOTO:

means a statement of target outcomes as agreed by the parties in accordance with clause 1.1;

According to clause 2.2 the Supplier must for each Target Outcome create:

Options that have the potential to achieve the Target Outcome. In respect of each Option the Supplier shall provide a forecast in terms of the estimated value, costs, risks and expiry date (if any) for that Option and shall inform the Customer of any requirements for execution of the Option and any circumstances in which the Option may expire.

Even after reading the overview it remains unclear exactly what the function of the Options are and what, if any, the work-flow for calling these are.

It appears odd that acceptance (adoption) of the Outcome Delivery is left to the Supplier but I assume this is considered reasonable given that one of the objectives of the Flexible Contract is to provide as much flexibility to vendor in deciding in how to achieve the SOTO. I don’t think any customer would be comfortable with this construct, especially if the Supplier’s payment hinges in part on achieving the Target Outcomes, which is one of the models (option B) suggested in the model SOTO schedule attached to the contract. Clause 2.7 isn’t much help in this context, as 2.7. (a) is stating the obvious and (b) lacks fundamental precision. As such in can easily be interpreted as granting the Customer defacto veto rights against any Outcome Delivery, which is hardly constructive either.

Related to this issue is the lack of consistency between clause 3.1 which states that:

In consideration of the provision of the Services by the Supplier, the Customer shall pay the fees in the amount and on the dates or with the frequency as set out in each SOTO.

and the “Hybrid Outcome-based fees” model in the model SOTO schedule, where payment is conditioned upon achieving the Target Outcomes.

Best Endeavours

I also find it fundamentally concerning that the contract relies on a unqualified “best endeavours” standard when defining the Suppliers responsibility under the contract. It is simply not good practice not to agree some further specification of what the parties intend “best endeavours” to mean. This can for instance be done by indicating what financial/resources the Supplier must expend in its attempt to fulfill its obligation. Failing to provide such guidance is to invite conflict and to introduce unnecessary uncertainty. While I applaud the authors stated aim of brevity this is not an area where one wants to sacrifice lack of precision for terseness. I have included a few links below that provides more in-depth information on the subject.

Intellectual Property Rights (IPR)

The IPR as specified in clause 5 model appears fairly sensible and reflects the reality of the deliverables being a composite of bespoke, vendor, third party proprietary and FOSS. That said the specific license granted to the Customer in respect of Supplier Software (clause 5.4) will in many cases have to be adapted to the needs of the Customers, both in respect of scope but also rights to create derived works etc. I would suggest moving the specification of this license to the SOTO schedule to facilitate this. This also applies to the specification of what types of software (Customised, Supplier, Third Party, Open Source) will constitute part of the Outcome Delivery, so the Customer may determine the feasibility of the rights obtained and clear any necessary rights.

While the contract does provide a warranty for non-infringement in respect of the Services and Outcome Delivery it does not address the issue of indemnification for such infringement, which any prudent customer will require.

The above is a summary of some of the issues I have identified with the current draft of the Flexlite contract. I summation I think the contract does a good job of capturing the idea of an outcomes focused contract, but unfortunately it does so at the expense of the customers position under the contract should things not go as planned.

I hope to have the opportunity to discuss Flexible Contracts with Gabrielle during goto Aarhus 2013 and do one or more follow posts on Flexible Contracts.

Links regarding best endeavours:

“All reasonable endeavours” a lesser obligation than “best endeavours”

Best and EffortsCase Analysis and Practical Guidance Under U.S. and U.K. Law

My delicious links about “best endeavours”

Full Disclosure: I am receiving a free GOTO ticket from the producers in exchange for blogging about the conference.

I put together my first draft itenary for GOTO Aarhus 2013. You can see it here.

Fortunately the programme did not pose too many dilemma’s for me to resolve, although choosing between “Obstacles And Patterns To Maximize Flow In IT Operations” and “Lambdas in Java: A peek under the hood” on Tuesday morning was a bit difficult.

I was especially pleased to see that David Nolen is giving a talk. David is fairly prolific in the Clojure community. He is the driving force behind core.logic and a significant contributor to ClojureScript. Core.logic is a implementation of miniKaren and provides robust logic programming facilities in Clojure and ClojureScript. As an example of how concise logic programming can be please see below Sudoku solver by David Nolen.

;; based on core.logic 0.8-alpha2 or core.logic master branch

(ns sudoku
    (:refer-clojure :exclude [==])
        (:use clojure.core.logic))

(defn get-square [rows x y]
    (for [x (range x (+ x 3))
          y (range y (+ y 3))]
              (get-in rows [x y])))

(defn init [vars hints]
    (if (seq vars)
        (let [hint (first hints)]
            (all
                (if-not (zero? hint)
                    (== (first vars) hint)
                        succeed)
                            (init (next vars) (next hints))))
                                succeed))

(defn sudokufd [hints]
    (let [vars (repeatedly 81 lvar)
          rows (- vars (partition 9) (map vec) (into []))
          cols (apply map vector rows)
          sqs  (for [x (range 0 9 3)
                     y (range 0 9 3)]
                     (get-square rows x y))]
                (run 1 [q]
                    (== q vars)
                    (everyg #(infd % (domain 1 2 3 4 5 6 7 8 9)) vars)
                    (init vars hints)
                    (everyg distinctfd rows)
                    (everyg distinctfd cols)
                    (everyg distinctfd sqs))))

;; ====

(comment
(sudokufd
    [0 0 3  0 2 0  6 0 0
     9 0 0  3 0 5  0 0 1
     0 0 1  8 0 6  4 0 0
     0 0 8  1 0 2  9 0 0
     7 0 0  0 0 0  0 0 8
     0 0 6  7 0 8  2 0 0
     0 0 2  6 0 9  5 0 0
     8 0 0  2 0 3  0 0 9
     0 0 5  0 1 0  3 0 0]))

gist

ClojureScript compiles Clojure into Javascript which is guarenteed to be digestable by the Goolge Closure compiler. While the ClojureScript implementation of core.logic is somewhat limited compared to the Clojure based one, you can still do some interesting things with it. If you are not familiar with ClojureScript you might want to check this tutorial out. Some interesting examples of using canvas / SVG with ClojureScript, can be found here

Full Disclosure: I am receiving a free GOTO ticket from the producers in exchange for blogging about the conference.