$ grep -r Tag: «backend»

After migrating (almost rewriting) my small project from Vapor 2 to Vapor 3 I've run benchmarks to compare performance. I didn't run benchmarks for the last version of Vapor 2.x so I will compare Vapor 2.1.0 to results of Vapor 3.1.0.

My server configuration:

• 2 GB RAM
• 1 CPU Core
• SSD
• 125 MBPS Out
• Ubuntu 16.04.2 LTS
• CouchDB

Benchmark from other server has been launched as:

wrk -t4 -c20 -d5m https://my_url

API just gets some data from CouchDB and returns it as JSON. Vapor project has been compiled with Swift 4.2.

Total requests

Requests per second

Average Latency

Memory usage

Vapor 3 has been rewritten and now it's based on Apple's NIO. It's faster than Vapor 2 but uses more memory which is not critical.

Previous benchmarks:
Benchmarks: Vapor 2 vs. Vapor 1
Benchmarks for Vapor 1.2.5
Swift Backend with CouchDB: Kitura vs. Vapor vs. Node.js

After migrating my pet project from Vapor 1 to Vapor 2 I've run benchmarks to compare performance. I didn't run benchmarks for the last version of Vapor 1.x which is 1.5.15 so I will compare Vapor 2.1.0 to results of Vapor 1.2.5 that I have from my last results.

My server:

• 2 GB RAM
• 1 CPU Core
• SSD
• 125 MBPS Out
• Ubuntu 16.04.2 LTS
• CouchDB

Benchmark from other server was launched as:

wrk -t4 -c20 -d5m https://my_url

API just gets some data from CouchDB and returns it as JSON. Vapor project has been compiled with Swift 3.1.1.

Total requests

Requests per second

Average Latency

Memory usage

Vapor 2 is about 14-15% faster than Vapor 1.2.5. And what's important - Vapor 1.2.5 memory usage grew up after benchmarks from 13.4 mb to 59.1 mb and Vapor 2 started with just 3.6 mb and its memory usage grew up to 5.3 mb. That's very impressive.

About a month ago I launched benchmarks for Server side Swift frameworks. Yesterday I updated my project to the new version of Vapor: 1.2.5. Previous version I used was 1.1.11.

One of the most important updates in Vapor 1.2.x is that Vapor is now using a non-blocking server.

So today I decided to run a new benchmark for my project updated to Vapor 1.2 and results surprised me. It's not only faster than the 1.1.x version, it's now equal to Node.js results!

Vapor vs. Node.js

My server:

• 2 GB RAM
• 1 CPU Core
• SSD
• 125 MBPS Out
• Ubuntu 16.10
• CouchDB

Benchmark from other server was launched as:

wrk -t4 -c20 -d5m https://my_url

API just gets some data from CouchDB and returns it as JSON.

Total requests

Requests per second

Average Latency

These results look much much better than previous, so I'm sure that the backend on Swift can be a good choice. The last question for me is memory usage now. Vapor grew after benchmarks from 13.4 mb to 59.1 mb. Maybe there's some memory leaks in my code, will see.

I made a few test projects to implement an API for my simple app to run some benchmarks. It just makes 1 request to Database and returns JSON data.

All projects was executed on Linode VPS in London:

• 2 GB RAM
• 1 CPU Core
• SSD
• 125 MBPS Out
• Ubuntu 16.10
• MySQL and CouchDB with equal data (about 13k rows/documents)

Benchmarks was run on another, more powerful dedicated server in Germany using wrk:

wrk -t4 -c20 -d5m https://my_url

Results are very different from other benchmarks by Ryan Collins.

What did I try:

• Node.js 7.0 + MySQL 5.7.16
• Node.js 7.0 + CouchDB 2.0 (via node-couchdb)
• Vapor 1.1.11 + CouchDB 2.0 (via HTTP)
• Kitura 1.1.1 + CouchDB 2.0 (via Kitura-CouchDB)

Total requests

Requests per second

Average Latency

Results are very disappointing for me. Node.js was 50% faster than Swift. It looks like it's still not the right time to make backends on Swift right now, unless you don't expect high load and just want to write in Swift.

UPDATE: after about a month I ran another tests for the new version of Vapor (1.2.5) vs. Node.js and Vapor was very very fast. So now I'm very optimistic about using Swift as a backend.

In the previous post I described my impressions of four Swift frameworks. More precisely, I wrote about three, and the fourth interested me, but I only got around to it now. In short — I am thrilled. It is wonderful.

First of all, the project has good documentation. Not perfect, but very good. With examples and learning materials. Everything in one place on the site — excellent.

First, the framework has a console tool that makes it very easy to create new projects, run existing ones, build them, deploy them to various services, and even create a Docker container. I think that is awesome, because whenever you start coding some project, even a test one, you immediately remember the various tools from the Node.js world and want to have something similar here.

Second, the set of libraries in the framework is excellent. There is an HTTP client, database wrappers, WebSocket support, templates; by default it offers a good project structure with everything organized into folders. Out of the box there is excellent support for JSON and data that can be converted both from JSON and into JSON. There is even a localization mechanism.

Third, I managed to make my own mini-client for CouchDB through plain HTTP requests, and it works. So far only for reading; I have not yet had time to check writing, but the fact that it all worked made me very happy.

Fourth, all requests are logged by default, roughly like in Express.js. So you set up routing and in the console you can see where requests are coming in, at least understanding what is happening. On top of that, the framework ships with a logging tool.

Fifth — my whole small test project compiled and ran on Linux without problems or crashes. It took a long time to compile, but it works, and that is great.

For memory usage it is worse than Perfect so far. Perfect started at 3.5 MB and grew to 5 MB. Vapor starts at 11 MB and grows to 14.9 MB. We will see what happens next; for now that is still twice less than Node.js.

For now I will probably stop at Vapor — in my opinion it is wonderful :)

A couple of months ago I came across a post about server-side Swift frameworks — a performance comparison of four Swift server frameworks against each other and against Node.js. Performance caught my attention. Judging by the results, Perfect came out on top. A month later the author posted another comparison of performance, but this time tested not on macOS, but on Linux. Perfect was in the lead again.

This weekend I decided to finally get my hands on it properly. Before that I took a quick look at the four frameworks being compared: Perfect, Kitura, Vapor and Zewo. I did not have many requirements for a framework; besides standard request routing and returning responses, I needed:

• Templating support, so it can return not only JSON, but HTML too.
• Support for network requests, since Foundation still has nothing ready for networking.
• Preferably ready-made libraries for working with different databases.

Naturally, all of this needs to work under Linux.

Kitura

Kitura is one of the most advanced frameworks. There are libraries for MySQL, CouchDB, MongoDB and others. The only problem is that the performance test results are disappointing — it trails behind. On the other hand, it is being built by mighty IBM, which not only develops it but also actively contributes to Swift itself. There seems to be a library for network requests. There is templating support too, but things are bad there. There is Mustache, which only works on macOS and was only moved to Swift 3 three days ago. There is also Stencil, but when I looked at it there was some trouble there as well, related to the fact that it still could not be used with the latest Swift and Xcode.

There are a ton of repositories on GitHub with different libraries and components for Kitura, while documentation is almost completely absent. Yes, there are examples, but they are all primitive; as soon as you realize you want to do something more complex, you hit a dead end. Maybe it can do all of that, but digging through the sources of a bunch of interdependent libraries feels like an unjustified waste of time.

Today I decided to make a test project with it. Seeing that they have a library for working with CouchDB, I wanted to check whether it works (because the one I wrote in an evening does not, more on that below), but swift build could not even resolve dependencies after I added the Kitura-CouchDB dependency, which depends on some logger, which depends on yet another logger. All from IBM. It is some kind of hell when all of it comes from one vendor and still does not work together. I opened an issue for them on GitHub, and that was the end of my attempt to code anything with it. Putting it aside for now.

They only started this in February of this year, so I hope they will polish everything properly; IBM certainly has the resources for that. I literally watched a video from a Swift conference today where they promote Swift backend development and their framework.

Update: To get the project to build, I needed to specify version 1.1 for the dependency in Package.swift, as they told me in response to my GitHub issue. Maybe something will work out now. Though the aftertaste remains.

Zewo

You open the GitHub page and once again see another primitive request-routing example. If you dig through the list of repositories, you see that there seems to be a PostgreSQL client, an HTTP client for network requests, and a bunch of wrappers around C libraries.

Again, there is no documentation, and its popularity is much lower judging by the number of stars on GitHub. When you visit the project website, you want to see documentation, not a one-page Hello World.

There is Mustache for templates, but it still has not been moved to Swift 3. It is unclear how actively it is being developed; there seem to be many components, but if they cannot keep up with new Xcode and Swift releases that shipped a month and a half ago, it does not inspire confidence. At all. You cannot even try it because of the lag behind.

Vapor

It looks presentable. There are libraries for MySQL, PostgreSQL, MongoDB, its own ORM. There is an HTTP client for requests. There is the Leaf template engine.

The documentation looks excellent. There are text and video tutorials on a separate site.

I have not had time to try it yet, but I really want to, despite the fact that it did not show the best results in the performance test.

Update: a post about Vapor.

Perfect

The most popular Swift framework so far, at least judging by the number of stars on GitHub. It was the leader in the performance test, so naturally this was the one I wanted to look at first.

The site has great documentation for everything it offers. Since it is all collected in one place on the site (and duplicated on GitHub), I immediately saw that it has a wrapper over cURL for network requests, database libraries for SQLite, MySQL, PostgreSQL, MongoDB, and its own ORM. This was the first framework where I saw everything I needed (because some of the others simply do not have decent documentation), so this is the one I decided to try.

How I Tried to Write a Backend in Perfect

Perfect handles basic routing perfectly. The syntax is simple. I immediately decided to try building an API with it. For the API I wanted to use CouchDB. Since they do not have a ready-made library for it, I decided to quickly sketch my own, because CouchDB can be used through plain HTTP requests, and the framework ships with a wrapper over C cURL.

In a couple of hours I sketched out a method for fetching data through a GET request. I tested everything on macOS and, hooray, it all worked. I implemented the methods I needed and decided to try writing something to the database as well.

That is where the pain started. On macOS the data simply would not write to the database. That is, I send a plain POST request:

let curlObject = CURL(url: url)

curlObject.setOption(CURLOPT_HTTPHEADER, s: "Content-Type: application/json; charset=utf-8")
curlObject.setOption(CURLOPT_POST, int: 1)
curlObject.setOption(CURLOPT_POSTFIELDS, s: jsonString)

curlObject.perform { code, header, body in
    // code
}

CouchDB kept returning an error saying that invalid UTF-8 JSON was being sent. I pasted the string I was sending to the database into a separate REST testing app and there the data was accepted just fine. The strangest part was that if I made many requests from my code, then 1–2 requests with exactly the same data would be accepted and written by the database, while the rest failed. I never understood what the problem was. Either someone has a bug, or I do not know how to use CURL.

I spent a long time fighting this problem, and in the end I decided to just forget about it and upload what I had already done to my Ubuntu Server. That is where the new adventures began.

Something that builds perfectly with the swift build command in the console on macOS may fail to build on Ubuntu. Moreover, something that builds in Xcode may fail to build in the console on macOS.

After spending a lot of time on trouble spots, mainly related to pulling data out of Dictionary values with several nesting levels, I finally made the code compile on Ubuntu.

It built, I sent a request — and the app on the server crashed. I never solved that problem either, and on top of that I was upset that something working perfectly on macOS simply crashes on Ubuntu, even though Swift 3.0.1 was used on both.

After that there is a feeling of disappointment. I spent so much time on it that I do not really want to give up.

But I did make one pleasant observation: essentially I rewrote the API for my app that I had previously done in Node.js. The Node.js API built with Restify uses 35 MB of memory. The same API written in Swift grew only to 5.2 MB of memory on macOS. On Ubuntu it starts at 3.5 MB, but I could not check whether memory consumption increases, because the binary simply crashes whenever you send a request.

Conclusion

Once again, we have to wait for new Swift versions and new framework versions to build something real with them. Also, at the end of October Apple formed a working group to improve tools for backends written in Swift, so there is hope. While writing this post I took another close look at Vapor and was very pleasantly surprised. That will be the next one I try.

So, in summary:

Kitura does not build and showed poor performance results, but IBM is actively working on it.

Perfect builds, but what I wrote does not work on Ubuntu even though it works on macOS; apparently we need to wait for new Swift and Perfect versions.

Zewo is lagging behind, and I am not placing much hope in it at all.

I urgently need to try Vapor. (Update: I did)

If your code compiles on macOS, that does not mean it will compile on Ubuntu. If it compiles on Ubuntu, that does not mean it will work there, even if it works on macOS.