One alternative way to use spec is spec-provider which is a library written by me. You can pass spec-provider multiple examples data (say 15 maps) and it tries to figure out the shape of the data, what's common between them, whether something is optional or not etc, and then describe that shape as a spec.
This tool is inspired by F#'s type providers which do something similar by looking at examples of JSON or by introspecting database schemas and use the gathered information to produce F# types in order to automate the process and remove some of that burden from the developer. Spec-provider is a bit more general in the sense that the data source is always EDN – it's just Clojure data structures in memory.
Of course you can use the inferred specs as normal, for example to generate even more data:
This is an example of spec-provider in action:
(require '[spec-provider.provider :as sp]) (sp/pprint-specs (sp/infer-specs [{:a 8 :b "foo" :c [:k :l]} {:a 10 :b "bar" :c ["k" "kk"]} {:a 1 :b "baz" :c ["k" "oo"] :d "boo"}] :toy/small-map) 'toy 's)
(s/def ::d string?) (s/def ::c (s/coll-of (s/or :keyword keyword? :string string?))) (s/def ::b string?) (s/def ::a integer?) (s/def ::small-map (s/keys :req-un [::a ::b ::c] :opt-un [::d]))
So if we ask spec-provider to infer a spec for these maps, we get the
::small-map
spec, where it looks like :a
, :b
, :c
are required keys (because
they appeared in all the examples of maps we passed), and :d
looks like it
isn't (because it was not present in all the cases). It has also detected that
:c
, is a collection of keywords or strings.
Spec-provider has some rules on how to infer specs, but they're obviously not perfect, so the idea is that you run it on your database, or files, or whathever other data you have and then you check the generated spec yourself and adjust it manually. It's essentially a development tool.
It could also be used for data inspection: The resulting spec is essentially a summary of the properties of your data. It would be possible to run spec-provider over the whole data of an ElasticSearch database (or any other mostly schemaless database) to find some anomalies you don't expect, like some key which you thought always contained a number and you may find that it sometimes is a string. Or some value that should never be nil, and you may find that in some cases it is. In fact, this is very close to how Dan Lebrero used spec-provider as described in his blog Production data never lies.
As promised, here is an example of the spec that we inferred above, generating more samples of data of the same shape:
(require '[clojure.spec.alpha :as s] '[clojure.spec.gen.alpha :as gen] '[net.cgrand.packed-printer :as ppp]) (s/def ::d string?) (s/def ::c (s/coll-of (s/or :keyword keyword? :string string?))) (s/def ::b string?) (s/def ::a integer?) (s/def ::small-map (s/keys :req-un [::a ::b ::c])) (ppp/pprint (gen/sample (s/gen ::small-map) 5))
({:a -1, :b "", :c ["" :g :g :s :+]} {:a 0, :b "", :c [:- :Q "H" "4" "w"]} {:a 0, :b "", :c ["3G" "j" "Hj" "" :Y :D "" :_i/+ :R9/H_ :?W/* :C "9l" "" "" "Zb" ""]} {:a 0, :b "Cdi", :c [:Q :e/n_ "" "" :l/G- :_ :n7/-f "I8C" :Df/+f :*6/KP :q/!p :? :A/_1 "32k"]} {:a -2, :b "88", :c [:*/?S :fX "OH" "" :b/- :YF :YI/s "4Q" "3"]})