I was actually looking at clipper.ai for model serving at my work. I know it's not 1-to-1 comparison as clipper is much more generic where as this seem to tie in closely with PyTorch. Can it support models created using other libraries?
Hi, thanks for the feedback, I'm the creator of the framework. Answering your question: I chose LevelDB because it offers a good transactional capability for the key-value store, it also provides a very easy-to-use embedded library that is very stable while providing a decent performance and a binary serialization that helps me to store protobuf-serialized content. In summary, it's due to a mix of stability, database lock, binary store and transactional support. I use it to store item metadata (still work on to expose to the APIs) and model features/predictions.
It was created by Christian Perone, a researcher at Polytechnique Montreal. I don't think he's active on HN. I saw he released it and just started looking through it.
It seems this solves a different problem from clipper.ai, which seems to store model parameters themselves to answer f(inbound object, model); this also stores data, so it can answer f(inbound object, thousands of existing objects, model) where f might be nearest-neighbor search.
Any reason why you chose LevelDB?
I was actually looking at clipper.ai for model serving at my work. I know it's not 1-to-1 comparison as clipper is much more generic where as this seem to tie in closely with PyTorch. Can it support models created using other libraries?