Bohr In

Check Out GitHub PRs Locally

Thu, 05 Mar 2020 14:16:26 +0000

I‘m a big believer in code reviews. When I’m trying to review a PR on GitHub, I almost always want to check out the code locally on my computer, so I can open it up in an IDE. This is easy to do, but unfortunately, it’s not commonly documented. You can always check the branch behind the PR out by name, but I find it easier to use the PR number. It’s also easier to do this as part of a larger script. Instructions to set this up are below.

Open the .git/config file from the root of the repo in a text editor. Find the section that looks like this (note that the value of url will be different for you):

[remote "origin"]
	url = git@github.com:surya-soft/Kedwig.git
	fetch = +refs/heads/*:refs/remotes/origin/*

Add this line (this will not change, regardless of what your url is):

    fetch = +refs/pull/*/head:refs/remotes/origin/pr/*

to the end of it, to make it look like so:

[remote "origin"]
	url = git@github.com:surya-soft/Kedwig.git
	fetch = +refs/heads/*:refs/remotes/origin/*
	fetch = +refs/pull/*/head:refs/remotes/origin/pr/*

Save the file.

Run git fetch. You will now have refs to each PR by number. To check out PR 1, you can run git checkout origin/pr/1. This will check out the PR in a detached head state. Replace 1 with the number of the PR you want to reivew, and you’re all set!

Swift On Server 2019 Edition

Tue, 31 Dec 2019 08:30:00 +0000

Last year, I did a talk at try! Swift NYC on the state of Swift on Server in 2018. Since then, Swift on Server has evolved quite a bit. This blog post is my opinion on where Swift on Server stands at the end of 2019.

Progress

In my talk last year, I pointed out several building blocks that were missing in the Swift on Server ecosystem. Of these, several have already been addressed. We now have Swift Log, Swift Metrics, Swift Prometheus, and several other building blocks in place.

IBM

The Swift Server Working Group recently announced that IBM will no longer be working on Swift on Server. It isn’t yet clear what will happen to Kitura, but I think it’s safe to assume Kitura’s days are numbered.

I believe this will be a substantial blow to the Swift on Server community. I had the good fortune to meet Chris and Ian at try! Swift. I’ve also had the privilege to get to know a few people who work on Swift on Linux right here in Bangalore. It’s unclear whether they will also be pulling back from the Swift community. These are great folks, and it’s rather unfortunate to hear that they will no longer be participating in the community.

However, I must admit, this isn’t entirely surprising. Kitura’s approach to Swift on Server appeared for the most part to be to go head to head with Java, NodeJS, etc. This always felt like a mistake to me. Given the limited resources behind Swift on Server, it was always going to be incredibly unlikely that Swift would ever catch up to the long standing & thriving ecosystems that surround other server side languages, like Java, Ruby, Python, JavaScript, etc.

So if Swift can’t really catch up to other ecosystems, why even bother trying?

Why Swift on Server?

The most common argument I’ve heard that is that iOS developers already know Swift, and it would be easy for them to transition to server programming. The second most common argument I’ve heard is that Swift on Server enables code sharing with iOS apps. In my opinion, these are both flawed arguments.

While the programming language being used to write server code is definitely a factor, it is a relatively insignificant factor. The problems & frameworks being used to solve those problems, ultimately are a lot more important. iOS developers aren’t not working on server code because they have to use Ruby, or Python. They’re not writing server code because most of the knowledge & experience they’ve gained is useless. All that UIKit knowledge you gained the hard way - totally useless. On top of this, you need to learn a ton about the server side ecosystem. How do you do deployments? How do you monitor in production? How do you deal with concurrency? How do you debug performance issues? Gaining this knowledge & experience is a lot harder than learning a new programming language. Therefore, my belief is that this reason was never going to drive Swift on Server adoption.

There are also multiple problems with the code sharing argument. First off, Swift still doesn’t officially run on Android, nor does it officially run on Windows, and as far as I’m aware, Swift’s WASM story still isn’t quite there. How many companies and teams do you know of that have only an iOS/macOS app, and a server, but no Android app, and no web app? It’s incredibly uncommon. Even if you’re on such a team that has only Apple front-ends, and a server component, what code would you realistically share? A few model objects? You could likely re-write those in a few hours or days. There’s not a lot of value in sharing this code. Besides, there’re better approaches to this sort of code sharing anyway: Protocol Buffers, gRPC, etc.

Having said all that though, I am still bullish on Swift on Server. However, for Swift on Server to gain adoption, I think it’s going to have to bring something to the table for folks who’re already server side developers today. I think Swift has a few things going for it that few other languages do when it comes to server side development.

Memory Safety Without Garbage Collection

Almost every popular server side language has both a VM/interpreter & garbage collection. While there are definitely advantages to both, they do have problems.

Garbage collected environments are unpredictable. It is quite common to end up with GC problems that are incredibly hard to reproduce. It’s also often hard to figure out whether you have a garbage problem, or a garbage collection problem. Using languages like C/C++ which do not have garbage collection comes with a major downside: lack of memory safety. Swift (due to its use of ARC), provides a fantastic middle ground - memory safe, and predictable.

Fast Startup Time

VMs are also quite slow to start. For example, it is not uncommon for a typical Java application to take several seconds (sometimes even minutes) to start up. This was a non-issue in the past, when you expected to have huge machines that ran processes which ran for days (even months), but in a cloud native world, this is a real problem. For example, in server less functions, a VM is a deathblow.

Performance

Swift is already quite fast ¹, compared to languages like Ruby & Python for most use cases. It is also faster than Java in some cases. However, what excites me about Swift is that it can become an order of magnitude faster than most other languages. Swift’s type system, value types (with copy on write), static dispatch, the String APIs in the standard library, etc. have the potential to make Swift compete with a well tuned C program on performance. Unfortunately, that potential is not yet realized. Swift 5 & 5.1 bring brought some huge improvements though, and I’m optimistic about Swift competing on performance with systems programming languages in the near future.

Swift NIO

In a year, Swift NIO has also moved forward quite a bit. I believe NIO is Swift on Server’s biggest asset. Vapor has already adopted it in a big way. Postgres-NIO has been accepted by the SSWG. There aren’t many (any?) robust non-blocking database drivers that are easy to use in other languages yet. NIO provides a base to build on - if the Swift on Server community can leverage it successfully to build high performance web frameworks, with non-blocking database drivers, I believe it will be an incredibly compelling reason for server side developers to start using Swift.

Competition

Rust

In my opinion, Rust is the biggest competitor to the Swift on Server ecosystem. Rust is incredibly fast, memory safe, and is not garbage collected. It makes very similar tradeoffs on many fronts. The tooling ecosystem around it is quite mature as well. Tokio is very similar to NIO, and has recently announced support for async/await, which will be fantastic for developer ergonomics when higher level frameworks support it. Existing Rust frameworks are already much faster than Swift. While the war isn’t over yet, Rust has won several battles.

Kotlin

The biggest hurdle with Swift & Rust, is that they are relatively recent, and the ecosystems around them are still quite immature. For example, if you want to write a Microsoft Excel file using Swift/Rust, there really isn’t a great library for that yet. If you need to connect to an Oracle or MS SQL database, there isn’t a good way to do that in Swift/Rust yet. Where Kotlin shines, is with its interop with Java. Being able to seamlessly integrate with Java (and the vast ecosystem around it) is incredibly powerful. This makes Kotlin fully viable as a server side language today for almost everyone. However, in the long run, unless Kotlin Native (and the ecosystem around it) emerges strongly, I believe Swift & Rust will bring substantial improvements to server side development to convince a lot of people to use them over Kotlin.

Wishlist

Below are a few things I would like to see make Swift on Server viable:

Performance on par with (or better than) Rust.
async/await. This is a huge boost to developer ergonomics when dealing with non-blocking code. A framework like Vapor would be so much nicer to use with async/await.
A solution/mitigation to fatalError bringing down the entire process, instead of just killing a single request. For example, accessing an element in array that is out of bounds will result in the entire process dying, which could result in thousands of requests being dropped.
Robust non-blocking database drivers for popular relational databases (including databases like Microsoft SQL Server and Oracle, which are not open source).
Equivalent of Diesel that supports all the above databases.
Robust connection pooling library.

Update on 2020-01-04

The folks at Vapor announced that Vapor Cloud is shutting down. I don’t know what implication this has on the Vapor project (if any).

If you have any thoughts, comments or concerns, please let me know! I’m @gopalkri on Twitter.

https://github.com/TechEmpower/FrameworkBenchmarks ↩

A Codable Tale of Woe

Wed, 26 Sep 2018 15:51:22 +0000

I recently encountered a somewhat unusual Swift problem. I had a struct that looked something like this:

struct Something {
    let id: Int
    let someValue: String
}

I wanted to be able to construct one of these from a JSON response coming from a server. Obviously, I figured the easiest way to do this was to use Codable. So I did:

struct Something : Codable {
    let id: Int
    let someValue: String
}

let decoder = JSONDecoder()
let decodedSomething = try decoder.decode(Something.self, from: jsonResponse)

Unfortunately, I then realized that this server was using different keys, and as a result the call to decode would fail. Here’s an example server response:

{
    "id": 1,
    "sV": "foo"
}

So, to deal with that, I did this:

struct Something : Codable {
    let id: Int
    let someValue: String

    enum CodingKeys: String, CodingKey {
        case id
        case someValue = "sV"
    }
}

let decoder = JSONDecoder()
let decodedSomething = try decoder.decode(Something.self, from: jsonResponse)

And… everything was hunky dory.

Except… it wasn’t. I had to also store a JSON representation of these structs on disk, and retrieve them later. I really did not want to use the same JSON representation as the server, because it looks ugly, and makes debugging annoying. I wanted to have the on-disk representation be:

{
    "id": 1,
    "someValue": "foo"
}

At first, I wasn’t really able to find a way to do this. It then occurred to me that I’d likely have to implement encode(to encoder: Encoder) and it’s counterpart init(from decoder: Decoder) to do this, so I attempted to. Unfortunately, that led me to another problem. I’d need to have different CodingKeys enums for the server representation, and for my on-disk representation. But, even if I did have different CodingKeys, how would I know which one to use with which Encoder or Decoder? This also led me to a broader realization. Even if I tried to make my on-disk representation a Plist, instead of JSON, I’d still have to use the same CodingKeys, which was suboptimal.

So, I did what every good developer does, and googled for possible solutions to this problem, but did not find any. Then, I started reading the documentation hoping I’d find a way to solve this problem, and eventually came up with this solution:

let decoderTypeKey = CodingUserInfoKey(rawValue: "decoderType")!
let encoderTypeKey = CodingUserInfoKey(rawValue: "encoderType")!

class ServerJSONDecoder : JSONDecoder {
    static let decoderType = "ServerDecoder"

    override init() {
        super.init()
        userInfo = [decoderTypeKey: ServerJSONDecoder.decoderType]
    }
}

class ServerJSONEncoder : JSONEncoder {
    static let encoderType = "ServerEncoder"

    override init() {
        super.init()
        userInfo = [encoderTypeKey: ServerJSONEncoder.encoderType]
    }
}

struct Something : Codable {
    let id: Int
    let someValue: String

    init(id: Int, someValue: String) {
        self.id = id
        self.someValue = someValue
    }

    enum CodingKeys: String, CodingKey {
        case id
        case someValue
    }

    enum ServerCodingKeys: String, CodingKey {
        case id
        case someValue = "sV"
    }

    init(from decoder: Decoder) throws {
        if let decoderType = decoder.userInfo[decoderTypeKey] as? String {
            if decoderType == ServerJSONDecoder.decoderType {
                let container = try decoder.container(keyedBy: ServerCodingKeys.self)
                id = try container.decode(Int.self, forKey: .id)
                someValue = try container.decode(String.self, forKey: .someValue)
                return
            }
        }
        let container = try decoder.container(keyedBy: CodingKeys.self)
        id = try container.decode(Int.self, forKey: .id)
        someValue = try container.decode(String.self, forKey: .someValue)
    }

    func encode(to encoder: Encoder) throws {
        if let encoderType = encoder.userInfo[encoderTypeKey] as? String {
            if encoderType == ServerJSONEncoder.encoderType {
                var container = encoder.container(keyedBy: ServerCodingKeys.self)
                try container.encode(id, forKey: .id)
                try container.encode(someValue, forKey: .someValue)
                return
            }
        }
        var container = encoder.container(keyedBy: CodingKeys.self)
        try container.encode(id, forKey: .id)
        try container.encode(someValue, forKey: .someValue)
    }
}

let serverDecoder = ServerJSONDecoder()
let something = try serverDecoder.decode(Something.self, from: jsonResponse)
print(something)
// Output: Something(id: 1, someValue: "foo")

let serverEncoder = ServerJSONEncoder()
print(String.init(data: try serverEncoder.encode(something), encoding: .utf8)!)
// Output: {"id":1,"sV":"foo"}

let encoder = JSONEncoder()
print(String.init(data: try encoder.encode(something), encoding: .utf8)!)
// Output: {"id":1,"someValue":"foo"}

The basic idea here is that every Encoder has a userInfo dictionary, that you can put anything into. That same userInfo dictionary is made available at the time of encoding or decoding. I choose to put in the “type” of encoder in there, and use that as a mechanism to determine the type of Encoder or Decoder. This is what the ServerJSONEncoder and ServerJSONDecoder subclasses do.

Then, in the init(from decoder: Decoder) and encode(to encoder: Encoder) methods I look for the existence of that key in the userInfo dictionary. If it exists, and has a value I’m looking for, I use the appropriate CodingKeys to encode or decode.

And… everything is hunky dory again.

Except… it isn’t.

One problem with this solution is that it if I add another type of encoder/decoder, I now also have to update the struct itself, which is not good.

Second, there’s a whole bunch of repetition in the implementations of init(from decoder: Decoder) and encode(to encoder: Encoder). I haven’t yet found a way to avoid this duplication. The reason is because the type of container changes based on which CodingKey is used. So, I can’t pull container out of the if. If you have any ideas on how to avoid this duplication, or another way to solve this problem, please let me know! I’m @gopalkri on Twitter.

Testing Paper

Tue, 01 May 2018 16:02:26 +0000

I recently came across this paper: “When, How, and Why Developers (Do Not) Test in Their IDEs” written by Moritz Beller, Georgios Gousios, Annibale Panichella, and Andy Zaidman.

Caveat: It doesn’t appear to be super recent ¹, the sample size is not large (< 500 developers), and their data is gathered from Eclipse ², so the conclusions drawn in the paper may need to be taken with a grain of salt. Having said that, I think the results are really interesting.

I recommend reading the paper, but I’ve gathered some of the interesting parts below (with my commentary):

developers rarely run their tests in the IDE

Woah, no beating around the bush here.

TDD is not widely practiced. Programmers who claim to do it, neither follow it strictly nor for all their modifications.

Seems obvious… TDD always seemed like a bit of a farce to me. There’s a good idea in there somewhere, but I think people take it way too far.

software developers only spend a quarter of their work time engineering tests, whereas they think they test half of their time

This one’s interesting - I always thought it was a 1:1 ratio. That is, if I spend 4 hours writing production code, I spend 4 hours writing tests. I’ll have to find a way to get more stats on this for my team.

Observed behavior often contradicted survey answers.

This is not at all surprising - this is why I believe in measuring real data, and not trusting that people can remember what they did.

The majority of projects and users do not practice testing actively.

Not surprising, but damning.

Developers largely do not run tests in the IDE. However, when they do, they do it heftily.

This doesn’t seem to be a bad thing though:

One reason why some developers do not execute tests in the IDE is that the tests would render their machine unusable, for example during the execution of UI tests in the Eclipse Platform UI project. The Eclipse developers push their untested changes to the Gerrit review tool and rely on it to trigger the execution of the tests on the continuous integration server. In such a scenario, the changes only become part of the “holy repository” if the tests execute successfully. Otherwise, the developer is notified via email. Despite the tool overhead and a possibly slower reaction time, our low results on test executions in the IDE suggest that developers increasingly prefer such more complex setups to manually executing their tests in the IDE. IDE creators could improve the continuous integration server support in future releases to facilitate this new workflow of developers.

Tests run in the IDE take a very short amount of time.

we found that testing in the IDE happens fast-paced. Most tests finish within five seconds, or less.

This is definitely a problem I’ve faced before. Nobody likes waiting on a computer.

Tests and production code do not co-evolve gracefully.

Not surprising, but pretty damning.

Developers frequently select a specific set of tests to run in the IDE. In most cases, developers execute one test.

This resonates with me. We use gradle to build, and IntelliJ as our IDE. Unfortunately they don’t play very nice with each other and occasionally, I can’t run tests from IntelliJ. This usually means I don’t run tests at all, because running one test from gradle is a pain.

Conclusion

Bottom line, this paper paints a pretty horrifying picture of testing in software development. I can’t say I’m surprised by this. Most code I’ve encountered isn’t tested properly, and yet most of it seems to mostly work. This isn’t to say that I think testing isn’t a worthwhile activity. I still do believe it’s a valuable tool. However, it is perhaps also worthwhile to think about what degree of testing is actually required instead of being dogmatic about TDD, BDD, etc.

Looks like it was published in 2015? ↩
Eclipse is a terrible IDE, so I wouldn’t expect it to make testing (or anything, really) easy. It’s also mostly only used for Java development. Perhaps other languages would have different results? ↩

Code Review On BitBucket

Mon, 23 Apr 2018 15:42:58 +0000

My team recently started using BitBucket to collaborate on a project. We didn’t make the choice - it was made for us. Historically, we’ve used GitHub. However, since we were going to be using it, I figured we might as well make the most of it. On the surface, BitBucket seems like a good deal. It’s got pipelines and deployments built in. Being made by Atlassian, you’d think that it integrates better with JIRA than any other source control system. It’s also significantly cheaper than GitHub. All of this sounded great, until… Code Review.

But, before I start complaining, a bit of background about our workflow. We have an organization on BitBucket. That organization has a repository (let’s call it Repo). Each person on the team, forks Repo, creates a branch on their fork, does some work, and files a PR. After this, at least one other member of the team has to review the PR, and provide constructive feedback. The reviewer(s) and author may go back and forth a few times until they both agree the PR is good to go. When things are good to go, we merge the PR, and we’re done.

This workflow worked great for us when we were on GitHub. There’s one huge problem with BitBucket though - you can’t seem to be able to check out a PR locally! BitBucket’s UI for review is fine, but it is no substitute for being able to look at the code in your IDE, with a debugger on standby. On GitHub checking out a PR locally was trivial. I was really quite baffled when the same process didn’t work on BitBucket.

So I went looking to Google, looking for help. I came across this post that suggested that I had to make a minor change to the GitHub way, and things would work. Cool! Except… Nope. Didn’t work. Turns out, that only works on an older version of BitBucket Server (previously called Stash). Ugh.

So not knowing what to do, I filed a support ticket to Atlassian asking them if they had a solution to this problem. The support person responded (very politely) with a link to this page. Unfortunately… that doesn’t work either, for several reasons.

They ask you to hover over a DOM element, wait for the browser to show the URL it points to, and use that for the next step. Wait what!? That sounds ridiculous!
Turns out, the DOM element that article refers to doesn’t even point to a link anymore.
Even if I could manage to predict the URL, the URL will be an https url, and we don’t use https to work with git, we prefer ssh.
Even if you’re willing to use https, you’d still need to have read access to the forked repo (which unlike GitHub, BitBucket does not set up by default when someone forks your repo).

Wow. Just… Wow.

Then I figured, there must definitely be some way of downloading a patch file of the PR? GitHub has a way to do this. Turns out, nope.

The Atlassian support person later pointed me to this issue and said they’d be fixing this issue soon. But, I looked a little closer at the issue, and it’s been open since, wait for it… 2012. 6 years! Good lord!

This is when I started seriously considering wasting the absurd amount of time it’d take to get us off BitBucket, and back on to GitHub. Before I gave up hope though, I figured it was worth looking into the BitBucket API to see if there was some way to get this to work. Fortunately, that did not let me down. There’s an endpoint that allows you to get a PR as a patch.

I set about trying to get this to work, and quickly ran across the hell hole that is OAuth2. I was about to give up again, when I noticed this section way at the bottom of the page:

Basic auth Basic HTTP Authentication as per RFC-2617 (Digest not supported). Note that Basic Auth with username and password as credentials is only available on accounts that have 2-factor-auth / 2-step-verification disabled. If you use 2fa, you should authenticate using OAuth2 instead.

Cool! I wasn’t using 2fa anyway, so time to get this working. After a bit of finagling, I came up with this snippet which I stuck into my ~/.zshrc:

function co_bb_pr() {
    curl -u "<USERNAME>:<PASSWORD>" https://bitbucket.org/api/2.0/repositories/<ORG>/<REPO>/pullrequests/$1/patch -L -o $1.patch && git apply $1.patch
}

I can now run:

co_bb_pr 27

And have a PR 27 checked out locally.

Finally.

If you have a better way of doing this, please let me know! I’m @gopalkri.

Rosco

Fri, 19 Jan 2018 08:10:00 +0000

Kotlin Data Class Constructor Problem Follow Up

Fri, 18 Aug 2017 09:53:00 +0000

I wrote about a problem I was having with Kotlin data class constructors a few days ago. I sent it to a few people and asked for thoughts on how they might address the issue.

A few people proposed a few different possible solutions to the problem. I thought I’d write them down here, as well as what I ended up doing.

Make constructor private

One proposed solution was to make the constructor private and instead have a factory method:

data class User private constructor (
    val userId: UUID,
    val name: String,
    val emailId: String
) {
    companion object {
        fun create(
            userId: UUID,
            name: String,
            emailId: String
        ) {
            return User(
                userId = userId,
                name = name.trim(),
                emailId = emailId.toLowerCase()
            )
        }
    }
}

On the face of it, this looks like a good solution. However, there is a problem with it that IDEA very nicely warns you about: the copy method.

Even if you make the constructor private, it is still exposed by the copy method that is generated by the compiler.

You could still write something like this:

val user = User.create(UUID.randomUUID(), "Foo", "foo@bar.com")
user.copy(emailId = "FOO@bar.com")

which is problematic.

Make fields private

Another proposed solution looks like this:

data class User(
    val userId: UUID,
    private var _name: String,
    private val _emailId: String
) {
    val name: String
        get() = _name.trim()

    val emailId: String
        get() = _emailId.toLowerCase()
}

There’re unfortunately a couple of problems with this as well. First, every time you read name, or emailId you end up creating a whole new object, which means more garbage. This seems unnecessarily wasteful.

Second, the call site becomes ugly:

val user = User(
    userId = UUID.randomUUID(),
    _name = "Foo",
    _emailId = "foo@bar.com"
)

Admittedly, this is not a problem if you don’t use named parameters. However, named parameters are super useful, especially when you have model objects (the obvious use case for data classes) with > 5 parameters. It significantly reduces the likelihood of making a mistake.

Eventual non solution

Eventually, I wasn’t able to find any solutions to this problem that still allowed me to use data classes. So I ended up not using a data class, and instead writing out all the boilerplate necessary:

class User private constructor(
    val userId: UUID,
    val name: String,
    val emailId: String
) {

    companion object {
        fun create(
            userId: UUID,
            name: String,
            emailId: String,
        ): User {
            return User(
                userId = userId,
                name = name.trim(),
                emailId = emailId.toLowerCase().trim()
            )
        }
    }

    override fun equals(other: Any?): Boolean {
        // Have IDEA generate this for me
        // Remember to re-generate it everytime I add a property to this class
    }

    override fun hashCode(): Int {
        // Have IDEA generate this for me
        // Remember to re-generate it everytime I add a property to this class
    }

    // Unfortunately IDEA cannot generate this for me, so type it out
    // I might eventually create a live template for this
    fun copy(
        userId: UUID = this.userId,
        name: String = this.name,
        emailId: String = this.emailId
    ): User {
        return create(
            userId,
            name.trim(),
            emailId.toLowerCase().trim()
        )
    }
}

Not ideal, but it works…

If you have a better solution in mind, please do let me know: @gopalkri!

Subscriptions

Thu, 17 Aug 2017 04:51:03 +0000

Recently, a lot of iOS and macOS apps have switched from a pay once and use until next major version pricing model to a subscription model. The latest example I’m aware of is Ulysses. I don’t use Ulysses, but one of the apps I use that has also made the switch is 1Password. There’s been a lot of discussion about this all over the internet.

Here’s a quick summary of both sides of the debate. Some people think subscriptions for iOS and macOS apps are good - they provide a way to increase revenue, and have it be more predictable. Other people are of the opinion that subscriptions do not make sense for “apps” - people are ok with paying a subscription fee for something like Dropbox, but not for an app like Ulysses. I thought that the discussion on the latest CoreInt episode put forth both sides of this debate much better than I can. I highly recommend giving it a listen.

However, I feel like there are some factors I haven’t seen discussed yet (perhaps they have been, and I just haven’t seen it).

Total cost of ownership

Most apps switching from a paid upgrades model to a recurring subscription model are also raising their total cost of ownership at the same time. Even if the amount of money you pay up front is lower, your total cost of ownership is way higher.

I’ve been a 1Password user for over 5 years now. During that time, I’ve spent around $75 on 1Password for Mac, and iOS. I use Dropbox to sync my passwords, but I don’t pay for Dropbox specifically for 1Password.

If you think about the total cost of ownership over 5 years, with 1Password’s subscription model, which is $2.99 per month (billed annually), that’s $180. That is more than 2 times my previous total cost of ownership!

So far, I haven’t really seen anyone make the switch from paid upgrades to subscriptions without also dramatically increasing the total cost of ownership. I’d be very interested to see if the backlash against subscriptions is as high if the total cost is no different, or lower.

The only things I can think of that reduce or maintain total cost of ownership are all content subscriptions. Things like Netflix, Apple Music, etc. If I watch 2 or 3 movies a month that are available on Netflix, I’ve reduced my price compared to renting them on iTunes (this is partly why I pay for Netflix). Having said that, content and software are really different beasts, so I’m not sure this really applies.

Subscriptions are good because developers won’t hold back features

The argument here is that in the paid upgrades pricing model, developers have to hold back features that are ready for the next major release in order to make paying for an upgrade worth while.

I think this has theory has already proven itself to be invalid. Great example: Adobe Lightroom. In the last 2 years, Lightroom has added virtually no meaningful new features. I’ve paid adobe $240 over that period for absolutely nothing.

This leads me to my next point…

Comparisons to Adobe, Office 365, etc.

A lot of people are pointing to subscriptions like Adobe’s Creative Cloud, and Microsoft’s Office 365 as successful switches to subscription based models. However, I believe this is a misleading comparison to make.

There are no alternatives to Adobe Creative Cloud, and Microsoft Office for the vast majority of people using it. Think about it - what would a Lightroom user switch to? What would a Photoshop user switch to? What would a Microsoft Office user switch to? You could say Apple Photos, Pixelmator, Acorn, or iWork, but really, in a professional context, none of those are real options. Even if you’re willing to switch, the people you work with won’t.

So even if Adobe and Microsoft do atrocious things like having a garbage installer, not allowing you to unsubscribe without making 12 phone calls, or charging you outrageous amounts of money for absolutely nothing in return, you’re still going to have to pay them - you simply have no choice.

Ulysses, 1Password, etc. however, are not in that category. For most users, there are loads of alternatives.

So when do subscriptions work?

I think Manton was on to something on CoreInt when he said people are willing to pay for apps with a server component because there are real costs to running servers. However, I’m not convinced that most people really understand the details of how software is built, and that servers cost money, etc. I like to think of it more as a difference between products and services.

Let’s take software out of the picture for a moment. Would you pay once a week to have someone clean your house? You would. Would you pay once a week to have a vacuum cleaner in your house? Highly unlikely!

In summary, I believe that subscriptions are great for services. They are a much harder sell for products.

Kotlin Data Class Constructor Problem

Tue, 15 Aug 2017 04:55:22 +0000

I’ve been trying to do something with a model class in Kotlin that I’ve done many times before in Java, and I’ve been left without a clean way to do what I need.

Here’s what I’m trying to do in java:

import org.jetbrains.annotations.NotNull;

import java.util.UUID;

class User {

    @NotNull
    private final UUID userId;

    @NotNull
    private final String name;

    @NotNull
    private final String emailId;

    User(@NotNull final UUID userId, @NotNull final String name, @NotNull final String emailId) {
        this.userId = userId;
        this.name = name.trim();
        this.emailId = emailId.toLowerCase();
    }

    @NotNull
    UUID getUserId() {
        return userId;
    }

    @NotNull
    String getName() {
        return name;
    }

    @NotNull
    String getEmailId() {
        return emailId;
    }

}

The key things to note are the calls to name.trim(), and emailId.toLowerCase() in the constructor. These calls ensure that every User object created has a “valid” value for name and emailId.

Initially, in kotlin, this is what I had:

data class User(
    val userId: UUID,
    val name: String,
    val emailId: String
)

The problem here is that I don’t see any way of putting in the calls to trim() and toLowerCase().

Do you know of any ways to solve this problem? Please let me know: @gopalkri!

An Interesting Swift Benchmark

Thu, 27 Jul 2017 06:45:56 +0000

Problem

I recently wrote about creating extensions on Optional<String>. When I shared this post in the Swift India slack group, one of the people there pointed out to me that I could just use the nil coalescing operator instead of the extension methods I’d created.

So:

fooLabel.text = bar.foo ?? ""

instead of:

fooLabel.text = bar.foo.orEmpty()

I had a real doh! moment for not thinking of this myself. I was going to update my original post to say “well actually, nevermind - just use the nil coalescing operator”. Then however, I got some more feedback, which is why I let the post remain.

One person told me that they didn’t even know Swift had a nil coalescing operator, and that orEmpty was obvious to them where ?? was not. Hmm, interesting. The fact that I knew the nil coalescing operator existed, but didn’t think to use it made me think that perhaps orEmpty was cleaner. FWIW, I’m still not sure what the right tradeoff to make is in this regard - use a language feature that may not be super obvious, or use something custom that probably won’t exist in other projects?

The more interesting piece of feedback I got though, was that the nil coalescing operator was slow in terms of build time. I believe it was based off of this post. While that sort of made some sense intuitively, I figured I’d write a quick benchmark to see if I could prove/disprove that theory.

The general strategy was to generate Swift source files that used each of these ways of doing the same thing and measure how long it took to compile.

Experiment

Here’s the quick and dirty python script I wrote to do this:

#!/usr/bin/python

import os
import time

common = '''
extension Optional {
    public func or(other: Wrapped) -> Wrapped {
        if let ret = self {
            return ret
        } else {
            return other
        }
    }
}
extension Optional where Wrapped == String {
    public func orEmpty() -> String {
        return self.or(other: "")
    }
}

struct Foo {
    let foo: String?
}
let foo = Foo.init(foo: "foo")
'''

N = [1, 100, 500, 1000, 2500, 5000, 10000]
M = 10


def benchmark(name, template):
    for n in N:
        content = '' + common
        for i in range(n):
            content += template.format(i)
        fn = os.path.expanduser('~/Desktop/foo.swift')
        with open(fn, 'w') as fout:
            fout.write(content)
        results = []
        for i in range(M):
            st = time.time()
            os.system('swiftc -o ' + os.path.expanduser('~/Desktop/foo') + ' ' + fn)
            et = time.time()
            results.append(et-st)
        write_csv([str(n), name] + [str(r) for r in results])


def write_csv(line):
    with open(os.path.expanduser('~/Desktop/results.csv'), 'a') as fout:
        fout.write(', '.join(line))
        fout.write('\n')

try:
    os.remove(os.path.expanduser('~/Desktop/results.csv'))
except FileNotFoundError:
    pass

benchmark('nil', '''
let a{} = foo.foo ?? ""
''')
benchmark('nil with type specified', '''
let a{}: String = foo.foo ?? ""
''')
benchmark('empty', '''
let a{} = foo.foo.orEmpty()
''')
benchmark('empty with type specified', '''
let a{}: String = foo.foo.orEmpty()
''')
benchmark('other', '''
let a{} = foo.foo.or(other: "")
''')
benchmark('other with type specified', '''
let a{}: String = foo.foo.or(other: "")
''')

Results

I ran it on my 15 inch 2017 Macbook Pro with a 2.8 GHz Intel Core i7 processor plugged into power, and fully charged (be warned: this takes a few hours to run - I let it run overnight). I used the swift compiler that ships with Xcode 8.3.3. I took the median of the 10 samples, and graphed them:

So, it turns out I was totally wrong - the nil coalescing operator is way faster than my ways.

However, I noticed something weird that the graph above hides:

You may want to open those images up in a new tab - they’re much higher res than is visible, but I was too lazy to change the markup of my page to actually use the resolution.

If you remove the 2500 and 5000 values, you’ll notice that orEmpty is faster than nil-coalescing.

Weird!

Nil coalescing is faster, but only if the number of times the compiler has to do it is larger.

I haven’t been able to figure out why this is so far, but I’ll keep digging. If you know why, or have any theories, please let me know: @gopalkri!

Conclusion

This benchmark is highly synthetic. First, it only uses one source file (hopefully your apps are not all in one source file). Second, I highly doubt that most code bases are going to have more than a few hundred of these occurrences. At that volume, and at these absolute times (0.6 - 1 second) per 1,000 occurrences, I don’t think performance is a reason to use one or the other. The question goes back to one of style, and I’m not really sure which way to go. At this moment in time, I’d probably go with the nil coalescing operator.

Extensions On Optional String In Swift

Fri, 21 Jul 2017 16:47:11 +0000

Earlier today I was writing some code that looked like this:

if let foo = bar.foo {
    fooLabel.text = foo
} else {
    fooLabel.text = ""
}
if let baz = bar.baz {
    bazLabel.text = baz
} else {
    bazLabel.text = ""
}

I figured there had to be a cleaner way to express this. My initial thought process was to write an extension to String that does this. Very quickly though, I realized I need to extend Optional, not String.

So I came up with this:

extension Optional {
    public func or(other: Wrapped) -> Wrapped {
        if let ret = self {
            return ret
        } else {
            return other
        }
    }
}

This allows me to now write:

fooLabel.text = bar.foo.or(other: "")

Much better!

Then I thought, there’s a way to make this even better for Optional<String> specifically. So I came up with this:

extension Optional where Wrapped == String {
    public func orEmpty() -> String {
        return self.or(other: "")
    }
}

And voila! I can now write:

fooLabel.text = bar.foo.orEmpty()

Way better!

Removing Passwords From PDFs

Thu, 13 Jul 2017 12:59:46 +0000

I recently came across a pdf file that required a password to open. This isn’t a super uncommon thing, particularly when sensitive files are being sent over email. The problem with this though is if I want to save the file (to say iCloud Drive), and open it later, I’m never going to remember the password when I need to open the file.

There’s usually a pretty simple way to “remove“ the password from pdf files. Unfortunately, for some reason that way did not work on this file.

I went looking around for another way to do this, and came across qpdf. And… it worked!

I figured I’d share this in case it’s useful to anyone else, and so I can find it easily when I need it the next time.

qpdf can be installed with homebrew: brew install qpdf. Once it’s installed, here’s how you use it to remove the password from a pdf file (in this example, a.pdf):

qpdf --decrypt --password=<password> a.pdf a-nopasswd.pdf

Why I Dislike CoreData

Wed, 26 Apr 2017 03:07:01 +0000

On a recent episode of Fatal Error, the hosts talked about why they’d never use Core Data in a new project. I’m one of those people as well. I’d definitely recommend listening to the episode, but I also thought I’d jot down my reasons for never using Core Data if I have a choice in the matter.

Magic

If I had to summarize my problems with Core Data in one word, it’d be this: magic. Core Data does a whole bunch of stuff that for all intents and purposes is black magic. Most of the time you have no idea how anything works. For instance, if you read a property on a model object, you don’t actually know whether it’s in memory, or is being faulted in from disk. Figuring it out is an extremely daunting task (often impossible, since Core Data is not open source), and I don’t know too many people who can actually use Core Data well.

Garbage API

IMO, CoreData’s API is garbage. Here’re the top reasons why.

Model objects need to subclass NSManagedObject

Your model objects need to subclass NSManagedObject. This means no more value types for model objects. It also means your code is forever tied to Core Data (see point below about isolation).

Everything is mutable

All your model objects are mutable, and can change underneath you in several circumstances. This means you have to reason about concurrency every time you do something as silly as a property read.

Need to pass contexts around everywhere

You basically cannot do anything without a context, so you have to pass contexts around everywhere, effectively making them global. In fact, I recently saw an app which sticks a global context in AppDelegate, and references it from a whole bunch of places in the app. This effectively makes the app untestable.

Hard to isolate

Good design encourages separation of concerns. Your data layer should not be intricately tied into your presentation layer. Ideally, your data layer should be defined by a protocol, and your presentation layer should simply treat that protocol as a dependency. If implemented this way, I could swap out my data layer from one implementation (say JSON files written to disk), to another (say SQLite), and have no impact on my presentation layer.

With CoreData however, this is virtually impossible, without also losing all the benefits of using CoreData. Swapping CoreData out for another persistence mechanism would require you to effectively re-write most of your app from scratch.

Okay, but I don’t ever want to swap out CoreData for something else. Why should I care? Because separation of concerns is a good idea anyway. It makes testing your app a heck of a lot easier, for one.

Massive learning curve

One of the things I like about Swift, is the idea of progressive disclosure. The idea is that there is a small learning curve to do simple things. As you go along your journey, you will learn more, as you need to. You shouldn’t have to climb a mountain just to get started.

The problem with CoreData, is that you need to learn a ton of stuff to just get started with it. Sure, there are a lot of well written tutorials about it, but the fact is, you shouldn’t need them.

Not portable

This one is not relevant if you’re only on Apple platforms. However, if you are also on Android, using SQLite on both iOS and Android allows you to not have to re-think a whole bunch of stuff again.

What about Realm?

I haven’t used Realm for anything significant yet. Realm fixes two major problems with CoreData: it is open source ¹, and portable.

However, the reason I chose not to delve into Realm too deeply, is that it has several of the problems above as well. Magic, mutability, passing around contexts, etc. I’m also not sure how solid their business model is, so there’s also a risk of the project being abandoned if their company shuts down.

So what do I actually use?

Currently, GRDB. It is a fantastically designed thin, Swifty API around SQLite. It doesn’t try to do too much. It allows you to define model objects the way you want, have them be value types, and have them be immutable. It supports writing SQL just as well as it supports its own SQL generation wrappers.

It does progressive disclosure quite well, IMO. To start with, write your model objects as you usually would, conform to the requisite GRDB protocols, and you’re good to go. If you need to do more, start writing SQL. But, if you don’t understand what WAL is, that’s totally fine - you probably don’t need it. If you do though, it’s right there for you to use.

If you’re writing in Objective-C, FMDB is pretty great.

I’ve also heard good things about YapDatabase, but I haven’t used it myself. Their API seems to be pretty nice, though.

Further reading: a Hacker News comment from Mike Ash on the topic.

Get in touch on twitter @gopalkri if you have any thoughts, comments, feedback, or questions!

Which means that at least in theory, you can go figure out what it’s doing. ↩

Multi Line Strings In Kotlin

Thu, 20 Apr 2017 05:13:16 +0000

I’ve recently been writing a fair amount of Kotlin for one of our products. We’re using it both in the Android app, as well as in the backend server. Since both of these use cases involve writing SQL, I looked around to see if Kotlin supports multi line strings, so I could write my SQL inline, but also not have it be littered with + and \n everywhere. Turns out, Kotlin is pretty great for this use case!

Consider this example from our Android code base:

val cursor = db.rawQuery(
        """
        SELECT
            ${TN.PROJECTS}.${CN.PROJECT_ID},
            ${TN.PROJECTS}.${CN.NAME},
            ${TN.PROJECTS}.${CN.COLOR_CODE},
            ${TN.PROJECTS}.${CN.CHANGE_TOKEN},
            ${TN.PROJECTS}.${CN.DELETED}
        FROM ${TN.PROJECTS}, ${TN.TIMERS}
        WHERE
            ${TN.PROJECTS}.${CN.DELETED} = '0' AND
            ${TN.TIMERS}.${CN.DELETED} = '0' AND
            ${TN.PROJECTS}.${CN.PROJECT_ID} = ${TN.TIMERS}.${CN.PROJECT_ID} AND
            ${TN.TIMERS}.${CN.START_TIME} BETWEEN datetime('now', '-14 days') AND datetime('now', '1 second')
        GROUP BY ${TN.PROJECTS}.${CN.PROJECT_ID}
        ORDER BY ${TN.PROJECTS}.${CN.NAME}
        """,
        null
)

TN and CN are enums which represent table and column names.

Ideally, I’d like to avoid all the ${} crap, but using those ensures I don’t make a typo and have my query fail. Aside from that caveat, this SQL is pretty readable, as it is appropriately indented, and logically broken up into multiple lines.

Best of both worlds, really!

Unfortunately, there is one problem with this code. You’re likely writing something like this in a situation where the statement is already indented 2-3 levels. If that’s the case, the resulting string is going to be littered with unnecessary indents because the lines after the first one do not begin on the first character of the line. So if you’re debugging this code, and you want to log the exact query string, or copy paste it into an sqlite3 (or your SQL client of choice) to play around with it, it’s going to be ugly.

Fortunately, Kotlin has this handled as well. Enter trimIndent! Stick a call to this handy little standard library function at the end of your multi line literal, and it will handle the rest.

For example, without trimIndent, the above query would look like this:

                    SELECT
                        PROJECTS.PROJECT_ID,
                        PROJECTS.NAME,
                        PROJECTS.COLOR_CODE,
                        PROJECTS.CHANGE_TOKEN,
                        PROJECTS.DELETED
                    FROM PROJECTS, TIMERS
                    WHERE
                        PROJECTS.DELETED = '0' AND
                        TIMERS.DELETED = '0' AND
                        PROJECTS.PROJECT_ID = TIMERS.PROJECT_ID AND
                        TIMERS.START_TIME BETWEEN datetime('now', '-14 days') AND datetime('now', '1 second')
                    GROUP BY PROJECTS.PROJECT_ID
                    ORDER BY PROJECTS.NAME
                                   

With trimIndent, it looks like this:

SELECT
    PROJECTS.PROJECT_ID,
    PROJECTS.NAME,
    PROJECTS.COLOR_CODE,
    PROJECTS.CHANGE_TOKEN,
    PROJECTS.DELETED
FROM PROJECTS, TIMERS
WHERE
    PROJECTS.DELETED = '0' AND
    TIMERS.DELETED = '0' AND
    PROJECTS.PROJECT_ID = TIMERS.PROJECT_ID AND
    TIMERS.START_TIME BETWEEN datetime('now', '-14 days') AND datetime('now', '1 second')
GROUP BY PROJECTS.PROJECT_ID
ORDER BY PROJECTS.NAME                   

Much better! 👍

There’s also a slightly different variant of this function called trimMargin which may suit your needs better.

Get in touch on twitter @gopalkri if you have any thoughts, comments, feedback, or questions!

Apple Accelerator

Thu, 06 Apr 2017 07:00:02 +0000

Recently, Apple launched a App Accelerator in Bangalore. Phil Schiller tweeted about it:

App Accelerator Bengaluru - now open for iOS developers https://developer.apple.com/accelerator/

I signed up for a couple of sessions on the first day of sessions:

Designing Great Apps
Advanced Swift

The sessions were held at Apple’s offices in RMZ Galleria.

A couple of people on the Swift India Slack asked me to take notes during the sessions and share them since not everyone who wanted to attend could attend. I figured I’d start with the first session in this post: Designing Great Apps.

Designing Great Apps

This talk was given by the resident “design guy”, Karthick Gopal. The goal of this talk was to highlight what makes great apps (was also referred to as “next generation” apps).

To start off with, ask yourself this:

What does your app make your users feel?

It’s not just about what features it has, what it looks like etc. It’s about what it makes users feel.

Great apps are:

Inviting. This is done with judicious use of colors, contrast, etc.
Accessible. 1 in 7 Indians has a disability. We should cater for them in our design. Use technologies like Dynamic Type to do this.
Engaging
Intuitive
Specialized
Personal
State of the art

They also stand out, and provide clear value to their users.

3 Steps To Make Great Apps

Use state of the art technologies
- Use the latest and greatest frameworks and extension points provided by Apple. However, do not blindly just integrate everything - think about what makes sense for your apps.
Optimize your apps
- Think about which devices in the Apple ecosystem your app makes (iPhones, iPads,  watch, Macs) sense to be on.
- Think about how your apps will work on the specific devices your apps will be on.
- As you design your app for a device, also think about whether your app could work on other devices in the future, even if that is not your current focus.
Specialize your apps
- Use  frameworks.
- Particularly extension points.
- However, do so judiciously.

Examples

Karthick highlighted a few apps in his presentation for having great design.

Ixigo

Ixigo has a fantastic on boarding experience, making it inviting for new users. It uses delightful animations to draw you in.

Bear

Bear focuses on the task at hand: taking notes. Icons and actions are very subtle. They use colors really well. It’s also fun, and engaging. There first note you see when you open the app has a cute bear in it. It also looks similar on iPhone, iPad, and Mac, while still staying true to each platform.

Mr. Crab 2

Mr. Crab 2 is a fun game. Not crammed with all kinds of buttons and features. Optimized for each platform it is on.

Not On The High Street

This is a premium shopping app in the UK. It is highly visual. Focus is on the item, and it’s photos. They have high fidelity photos everywhere. Very little text.  TV app is video intensive, since video is most used on  TV. However, since people are not used to buying stuff on their TV, it uses handoff to allow the user to purchase an item being viewed on the  TV on their iPhone.

Night Sky 4

Night Sky 4 has fantastic user interaction. Super smooth, and performant. Does not distract you with information you don’t need until you ask for it.

Field Day

Field Day is a game designed specifically for  Watch. Designed for kids, so has a lot of cute animations, transitions, and graphics.

Summary

Karthick summarized with these points:

Have a great on boarding experience
Exhibit a clear and precise design language
Leverage key iOS technologies
Exist on the platforms where your users do
Surprise and delight users over time
Deliver clear value

Overall, I quite enjoyed the session, and learned a few things from it. Hopefully this is useful to the people who weren’t able to attend this time around. Feel free to hit me up on twitter @gopalkri if you have any thoughts, comments, feedback, or questions!