Analyzing Alloy Constraints Using an Smt Solver: a Case Study

Analyzing Alloy Constraints using an SMT Solver: A Case Study
Aboubakr Achraf El Ghazi
Karlsruhe Institute of Technology Karlsruhe, Germany

Mana Taghdiri
Karlsruhe Institute of Technology Karlsruhe, Germany

[e-mail] ABSTRACT
This paper describes how Yices, a SAT Modulo theories solver, can be used to analyze the address-book problem expressed in Alloy, a first-order relational logic with transitive closure. Current analysis of Alloy models – as performed by the Alloy Analyzer – is based on SAT solving and thus, is done only with respect to finitized types. Our analysis generalizes this approach by taking advantage of the background theories available in Yices, and avoiding type finitization when possible. Consequently, it is potentially capable of proving that an assertion is a tautology – a capability completely missing from the Alloy Analyzer. This paper also reports on our experimental results that compare the performance of our analysis to that of the Alloy Analyzer for various versions of the address book problem.

[e-mail]
SAT via bit blasting, they can be analyzed with respect to only a few bits. Consequently, Alloy offers limited support for arrays and numerical constraints. This motivated our project: to analyze Alloy models using an SMT solver rather than a SAT solver. SMT solvers are particularly attractive because they can efficiently prove a rich combination of decidable background theories without sacrificing completeness or full automation. Furthermore, their capability to generate satisfying instances as well as unsatisfiable cores[23] (offered only by some SMT solvers) supports Alloy’s lightweight and easy-to-use approach. This paper describes the first step of our project. It reports on a case study where an SMT solver, namely Yices[7], is used to analyze an Alloy model, namely the address book problem[16]. To our knowledge, this is the first attempt to analyze a rich relational logic using an SMT solver. We have checked

References: [1] The HOL Light theorem prover. http://www.cl.cam.ac.uk/ jrh13/hol-light/. [2] Isabelle. http://www.cl.cam.ac.uk/research/hvg/Isabelle/. [3] KeY project: Integrated deductive software design. http://www.key-project.org/. [4] Mondex case study with alloy. http://www.eleves.ens.fr/home/ramanana/work/mondex. [5] PVS specification and verification system. http://pvs.csl.sri.com/. [6] The satisfiability modulo theories library (smt-lib). http://goedel.cs.uiowa.edu/smtib. [7] Yices: An SMT solver. http://yices.csl.sri.com/. [8] A. Armando, J. Mantovani, and L. Platania. Bounded model checking of software using SMT solvers instead of SAT solvers. STTT, 11(1):69–83, 2009. [9] D. Barsotti, L. Nieto, and A. Tiu. Verification of clock synchronization algorithms experiment on combination of deductive tools. ENTCS, 145:63–78, 2006. [10] M. Botincan, M. Parkinson, and W. Schulte. Separation logic verification of c programs with an SMT solver. ENTCS, 254:5–23, 2009. [11] G. Dennis, F. Chang, and D. Jackson. Modular verification of code with SAT. In ISSTA, pages 109–120, 2006. [12] B. Dutertre and L. de Moura. The yices SMT solver. Available at yices.csl.sri.com/tool-paper.pdf, 2006. [13] L. Erk¨k and J. Matthews. Using yices as an o automated solver in Isabelle/HOL. In AFM, 2008. [14] S. Ghilardi and S. Ranise. Model checking modulo theory at work: the intergration of yices in MCMT. In AFM, 2009. [15] D. Jackson. Software Abstractions: Logic, Language, and Analysis. The MIT Press, 2006. [16] D. Jackson. Software Abstractions: Logic, Language, and Analysis. Pages 5-23, The MIT Press, 2006. [17] E. Kang and D. Jackson. Formal modeling and analysis of a flash filesystem in alloy. In ABZ, 2008. [18] S. Khurshid. Generating Structurally Complex Tests from Declarative Constraints. PhD thesis, MIT, 2003. [19] R. Leino and R. Monahan. Reasoning about comprehensions with first-order SMT solvers. In SAC, pages 615–622, 2009. [20] S. Narain, G. Levin, V. Kaul, and S. Malik. Declarative infrastructure configuration synthesis and debugging. In JNSM, 2008. [21] M. Taghdiri and D. Jackson. Inferring specifications to detect errors in code. JASE, 14(1):87–121, 2007. [22] M. Vaziri. Finding Bugs in Software with a Constraint Solver. PhD thesis, MIT, 2004. [23] L. Zhang and S. Malik. Validating SAT solvers using an independent resolution-based checker. In DATE, pages 10880–10886, 2003. 6. CONCLUSIONS We have described a case study in which the Alloy address book problem is analyzed using the Yices SMT solver. This case study is the first step of a bigger project in which the Alloy language will be translated to the Yices input language automatically, and thus, analyzed using an SMT solver rather than a SAT solver. The main advantage of this approach is to avoid finitization of types when possible, and thus to provide a stronger analysis than the one offered by the current Alloy Analyzer. Since the Alloy Analyzer translates all language constructs to propositional logic and uses a SAT solver, it performs the analysis only with respect to a finite scope. It can never prove that an assertion is a tautology, even for the simplest models. Unlike SAT solvers, SMT solvers support a number of theories and can prove or refute the constraints within those theories without sacrificing completeness. Our case study was a witness to feasibility of this project. Out of a total of 9 assertions that were checked in 3 models, 5 were proven to be tautologies. That is, all Alloy constraints could be translated to the Yices without having to finitize any type. In the other 4, finitization was required, but only for a subset of the types; the others were left infinite. The Alloy logic is undecidable. Therefore, finitization is inevitable. Our case study, however, showed that the finitization can be done on-demand; only for those types to which certain language constructs are applied (e.g. universal quantifiers and transitive closure). During this case study, we realized that many of the Alloy constructs can be translated to Yices in more than one way. So far, we have picked the ones that work reasonably well for the address book problem. More experiments are needed until we can fix a particular translation rule for each Alloy construct. In fact, our current experiments show that our translation of transitive closure is too difficult to analyze when it is applied to different relations and then checked for equality. We are currently investigating other translation techniques to mitigate this problem. Although, in our experiments, any time that Alloy did not find a counterexample, Yices did not either, this is not always the case. Some Alloy models for which the analyzer cannot find a counterexample, actually have counterexamples, but in higher scopes than the one that the analyzer can check. Investigating whether Yices can analyze such models in a high-enough scope to find the counterexample, will be