Merge pull request #332 from avsm/master

release 0.17.1

CHANGES

@@ -1,3 +1,10 @@
+0.17.1 (2015-04-24):
+* [async] Limit buffer size to a maximum of 32K in the Async backend
+  (#330 from Stanislav Artemkin).
+* Add `Cohttp.Conf.version` with the library version number included.
+* Remove debug output from `cohttp-curl-async`.
+* Add the beginning of a `DESIGN.md` document to explain the library structure.
+
 0.17.0 (2015-04-17):
 
 Compatibility breaking interface changes:

DESIGN.md

@@ -0,0 +1,71 @@
+Cohttp is designed to be an HTTP implementation that is an "onion", with the
+portable parsing core progressively introducing I/O, and then higher-level
+abstractions for various HTTP operations.  Here's a description of each layer:
+
+- The very first layer (in `lib/`) is a pure OCaml, non-blocking layer that
+  handles simple parts of the HTTP protocol such as parsing requests and
+  responses, various header parsers (e.g. cookies) and codes.
+
+- Some layers of HTTP need some notion of I/O, and so there is a set of
+  signatures in `lib/s.mli` that defines some common module types that can be
+  used to build parameterised modules (also known as functors).  The first one
+  used in the `lib/` layer is the IO module type, which defines the minimal
+  collection of functions used by cooperative threading libraries.  The pure HTTP
+  core uses this IO module to capture IO-based operations, such as Transfer_IO
+  (for transfer encoding).
+
+- There are three implementations that satisfy the IO module in the tree: Lwt,
+  Async and String.  The first two are full cooperative threading libraries,
+  and the latter is used by the js_of_ocaml backend to read/write between
+  Strings.
+
+- Now that IO has been handled, we can send HTTP requests and responses from
+  Lwt or Async.  However, at this point some differences appear in the
+  implementations of Async and Lwt, notably in how they handle cancellation of
+  threads and also higher-level iterators (e.g. Async has Pipes, and Lwt has
+  Lwt_stream -- both quite different).  Therefore, we build backend-specific
+  Client and Server modules that use their respective threading libraries in as
+  native a style as possible, but still reusing the core HTTP library from
+  `lib/`.  These can be found in `Cohttp_lwt` and `Cohttp_async` respectively.
+  Dave Scott also wrote an (as yet not merged) POSIX blocking version that they
+  use in the XenAPI daemon.
+
+- Lwt comes with an additional twist -- it is portable to both Unix *and* the
+  MirageOS, which has no Unix at all!  Lwt makes it possible to define a "Lwt
+  core" that uses the portable Lwt thread abstractions, but doesn't use any
+  OS-specific functionality.  Thus we can define an HTTP Client and Server in
+  Cohttp_lwt, but still not tie ourself to one particular OS.  This Cohttp_lwt is
+  then used by the Cohttp_lwt_unix and Cohttp_mirage backends to hook it into the
+  operating system.
+
+- There's no commonality at present between Cohttp_async and Cohttp_lwt, but
+  that's the topic of a design discussion at the moment.  It should be possible
+  to build a common signature between the two. (TODO add issue)
+
+- Andy Ray did something interesting with the Lwt backend: he ported it to
+  _javascript_ by implementing an IO backend that marshals the requests to and
+  from strings.  This allows REST API users built over Cohttp (such as
+  ocaml-github) to compile to pure _javascript_ as well.
+
+Drawbacks:
+
+- The heavy use of functors does make it hard to navigate the 'end user' API,
+  even though those interfaces never expose any functors (for instance, you
+  just use Cohttp_lwt_unix directly in most cases). This is a drawback of current
+  OCaml tooling, and Merlin (for IDEs) and Codoc (for cross-referenced
+  documentation) will fix this soon.
+
+- A bigger problem that needs to be addressed in Cohttp2 is body handling,
+  which we basically got wrong in this iteration.  The Body module is not
+  idempotent, so to_string does not always return the same value if called
+  multiple times.  The caller can currently be careful, but this is just an awful
+  part of the API.  There are enough users of Cohttp that we'll leave it for 1.0,
+  but hopefully fix it quite rapidly for 2.0.
+
+- Cohttp is not a complete HTTP client, and doesn't implement the full logic
+  for redirections, loop detection and so on.  That's the job of a library
+  built over it, and there is some nascent code in
+  [opam-mirror](http://github.com/avsm/opam-mirror) that can do this.  Before
+  building this, David Sheets and I want to look at some of the more larger API
+  clients built using it (such as Vincent Bernardoff's BitStamp API)
+  and take a shot at a portable client API that will work with both Lwt and Async.

_oasis

@@ -1,6 +1,6 @@
 OASISFormat: 0.4
 Name:        cohttp
-Version:     0.17.0
+Version:     0.17.1
 Synopsis:    HTTP library for Lwt, Async, JavaScript and Mirage
 Description: CoHTTP is an OCaml library for creating HTTP clients and
   daemons.  It has a portable HTTP parser.  Optional dependencies

bin/cohttp_curl_async.ml

@@ -31,7 +31,7 @@ let make_net_req uri meth' () =
    show_headers (Cohttp.Response.headers res);
    body
    |> Body.to_pipe
-   |> Pipe.iter ~f:(fun b -> prerr_endline ("XX " ^ b); return ())
+   |> Pipe.iter ~f:(fun b -> print_string b; return ())
 
 let _ =
   let open Command.Spec in

bin/cohttp_curl_lwt.ml

@@ -92,7 +92,7 @@ let cmd =
     `P "$(b,curl)(1), $(b,wget)(1)" ]
   in
   Term.(pure run_client $ verb $ ofile $ uri $ meth),
-  Term.info "cohttp-curl" ~version:"1.0.0" ~doc ~man
+  Term.info "cohttp-curl" ~version:Cohttp.Conf.version ~doc ~man
 
 let () =
   match Term.eval cmd

bin/cohttp_proxy_lwt.ml

@@ -103,7 +103,7 @@ let cmd =
         on the issue tracker at <https://github.com/mirage/ocaml-cohttp/issues>";
   ] in
   Term.(pure lwt_start_proxy $ port $ host $ verb $ ssl_cert $ ssl_key),
-  Term.info "cohttp-proxy" ~version:"1.0.0" ~doc ~man
+  Term.info "cohttp-proxy" ~version:Cohttp.Conf.version ~doc ~man
 
 let () =
   match Term.eval cmd with

bin/cohttp_server_lwt.ml

@@ -165,7 +165,7 @@ let cmd =
         on the issue tracker at <https://github.com/mirage/ocaml-cohttp/issues>";
   ] in
   Term.(pure lwt_start_server $ doc_root $ port $ host $ index $ verb $ ssl_cert $ ssl_key),
-  Term.info "cohttp-server" ~version:"1.0.0" ~doc ~man
+  Term.info "cohttp-server" ~version:Cohttp.Conf.version ~doc ~man
 
 let () =
   match Term.eval cmd with

lib/META

@@ -1,6 +1,6 @@
 # OASIS_START
-# DO NOT EDIT (digest: 9d152d2560e713c2454d84c04d50f2bb)
-version = "0.17.0"
+# DO NOT EDIT (digest: 83cf05ab603ed7321121aed3976a1f00)
+version = "0.17.1"
 description = "HTTP library for Lwt, Async, JavaScript and Mirage"
 requires =
 "re.emacs stringext uri uri.services fieldslib sexplib bytes base64"
@@ -10,7 +10,7 @@ archive(native) = "cohttp.cmxa"
 archive(native, plugin) = "cohttp.cmxs"
 exists_if = "cohttp.cma"
 package "lwt-unix-test" (
- version = "0.17.0"
+ version = "0.17.1"
  description = "HTTP library for Lwt, Async, JavaScript and Mirage"
  requires = "cohttp.lwt unix lwt.unix oUnit"
  archive(byte) = "cohttp_lwt_unix_test.cma"
@@ -21,7 +21,7 @@ package "lwt-unix-test" (
 )
 
 package "lwt-core" (
- version = "0.17.0"
+ version = "0.17.1"
  description = "HTTP library for Lwt, Async, JavaScript and Mirage"
  requires = "lwt uri cohttp"
  archive(byte) = "cohttp_lwt.cma"
@@ -32,7 +32,7 @@ package "lwt-core" (
 )
 
 package "lwt" (
- version = "0.17.0"
+ version = "0.17.1"
  description = "HTTP library for Lwt, Async, JavaScript and Mirage"
  requires = "cohttp.lwt-core unix lwt.unix conduit.lwt-unix magic-mime"
  archive(byte) = "cohttp_lwt_unix.cma"
@@ -43,7 +43,7 @@ package "lwt" (
 )
 
 package "js" (
- version = "0.17.0"
+ version = "0.17.1"
  description = "HTTP library for Lwt, Async, JavaScript and Mirage"
  requires = "cohttp.lwt-core js_of_ocaml"
  archive(byte) = "cohttp_lwt_xhr.cma"
@@ -54,7 +54,7 @@ package "js" (
 )
 
 package "async" (
- version = "0.17.0"
+ version = "0.17.1"
  description = "HTTP library for Lwt, Async, JavaScript and Mirage"
  requires = "uri cohttp threads async conduit.async magic-mime"
  archive(byte) = "cohttp_async.cma"

lib/conf.ml

@@ -15,4 +15,5 @@
  *
  *)
 
-let user_agent = "ocaml-cohttp/0.17.0"
+let user_agent = "ocaml-cohttp/0.17.1"
+let version = "0.17.1"

lib/conf.ml.ab

@@ -16,3 +16,4 @@
  *)
 
 let user_agent = "ocaml-cohttp/${pkg_version}"
+let version = "${pkg_version}"

lib/conf.mli

@@ -1,5 +1,6 @@
 (*
  * Copyright (c) 2015 Christophe Troestler <Christophe.Troestler@umons.ac.be>
+ * Copyright (c) 2015 Anil Madhavapeddy <anil@recoil.org>
  *
  * Permission to use, copy, modify, and distribute this software for any
  * purpose with or without fee is hereby granted, provided that the above
@@ -19,3 +20,6 @@
 
 val user_agent : string
 (** User-Agent, including the version of this library. *)
+
+val version: string
+(** The version number of this library. *)

opam

@@ -2,7 +2,7 @@ opam-version: "1.2"
 
 name: "cohttp"
 maintainer: "anil@recoil.org"
-version: "0.17.0"
+version: "0.17.1"
 authors: [ "Anil Madhavapeddy"
            "Stefano Zacchiroli"
            "David Sheets"

setup.ml

@@ -1,7 +1,7 @@
 (* setup.ml generated for the first time by OASIS v0.4.5 *)
 
 (* OASIS_START *)
-(* DO NOT EDIT (digest: 0361a9cd210c505c1a17dace6c7ba25d) *)
+(* DO NOT EDIT (digest: 7840ff0fea77d9f849727cb97b6cdb3e) *)
 (*
    Regenerated by OASIS v0.4.5
    Visit http://oasis.forge.ocamlcore.org for more information and
@@ -6999,7 +6999,7 @@ let setup_t =
           alpha_features = [];
           beta_features = [];
           name = "cohttp";
-          version = "0.17.0";
+          version = "0.17.1";
           license =
             OASISLicense.DEP5License
               (OASISLicense.DEP5Unit
@@ -8686,7 +8686,7 @@ let setup_t =
        };
      oasis_fn = Some "_oasis";
      oasis_version = "0.4.5";
-     oasis_digest = Some "\024A\212\196\233 \219c_\195\227\132\131U\019\144";
+     oasis_digest = Some "\148ëj{\137:Ó`>\\\007\023k±\006ó";
      oasis_exec = None;
      oasis_setup_args = [];
      setup_update = false