Plone fest continues here in Naples. Sally Kleinfeldt from the Nature Conservancy
is presently giving a talk about the ways their organization approached Plone development for their custom application development.The Nature Conservancy
is an environmental nonprofit protecting ecologically important lands and waters. They take a science-based approach and have protected 117 million acres. They have 1 million members, work in all 50 states and 30 countries and have 3,000 employees.
They have a traditional IT shop, supporting different departments, They also create custom applications for the conservation work. You can't buy these sorts of things off the shelf. They adopted Zope in 2000 to develop Nature.org and their intranet and they did their Zope development through the web.
In 2004 they moved to Plone for ConserveOnline.org (public community sharing portal). They did filesystem development with RDB or ZODB storage.
They build spatial and non-spatial apps with a mix of data and content. Users, roles and permissions are important. Workflow is sometimes important. There was an increasing need for custom applications for conservation and cross-application integration.
Example 1 - The Wrong Way
ConPro - a system to manage structured and unstructured information about their conservation projects.
Information was previously managed in complex Excel spreadsheets and associated documents, often in the field. There was a need for import, export, versioning, locking, security and search.
A large requirements phase was undertaken. There were 50+ pages of functional requirements - and this took 6 months to make. There was an automatic assumption of relational storage for ad hoc SQL queries. A spreadsheet was made for this project, and that spreadsheet dictated the data model. End users defined edit screen mockups based on the spreadsheet, and they wanted it to "work like Excel". The requirements for versioning, security and workflow were fairly vague at this point as the users weren't really sure what they wanted or needed.
The developers knew they wanted to work with Plone for its compatibility with the ConserveOnline project. However, they were unable to hire an expert consultant to guide development (due to internal constraints) and they didn't have a lot of internal knowledge. The team they put together put a couple months of efforts into developing the spreadsheets, import/export, SQL methods, and edit forms. The design of classes, security, overall application were left until the end.
Archetypes was used for object security (stub objects). The rest was hand-crafted SQL methods and page templates. They didn't use workflow. Of course, the users changed their minds in the middle about the data model. They found a couple Plone experts to come in and help get it done. Results
: The project was late and overbudget. It took 30 person-months of effort. They produced roughly 400 hand-written SQL methods. They wrote 50 hand-written edit, display, report forms. Ad hoc SQL queries, which was supposed to be a major feature and the primary reason relational storage was chosen, was never implemented (and nobody ever cared or complained). The application was really difficult to maintain and extend. But.... this was a BIG success with users! Lessons Learned
* Avoid an excessive requirements phase for new/unclear apps
* Users should not dictate technology decisions
* Relational storage increases cost, avoid if possible
* Have guidance from experts on your first project, learn to do things The Plone Way
* Start with classes, security, overall application functioning; page templates later
Example 2: A Better Process
Ecoregional Assessment Status Tool (ERA)
This is a science-based analysis of an ecoregion to determine the highest value areas for conservation. They wanted a new system for management to track the status of this work, with around 200 assessments, each with roughly 20 attributes.
Unlike the previous example, this was a well-understood application. The initial version had been done in pure Zope prior to ConPro. An informal project was driven by the developer to re-implement in The Plone Way. The customer was very flexible and available, and no technology assumptions were made. It was a relatively simple application.
They used Archetypes content types with ZODB storage. UML modeling was performed with ArchGenXML to generate their starter project. Results
* 1 week to develop!
* 2 content types, 23 page templates, workflow to track status
* Easy to maintain and extend
* Big success with users!
Customer: "This has been my most pleasant development experience ever." Lessons Learned
* Technology choices had been haphazard
* They needed to base decisions on what works in their environment
* Repeatable development results Application Patterns
* A set of components, technologies, and tools
* Used to create a certain type of application
* Good at solving certain types of problems
* In the context of their organization and level of development expertise Definition
* Internet or intranet
* Type of information (spatial, structured, unstructured)
* Sensitivity of information
* Amount of information
* Number of users, sessions
* Org level (global, regional, local)
* Community (conservation, marketing, finance, legal, etc.) Components
GIS, web, database, ORM, search, authentication, user management, security, workflow, error handling, logging, help system, etc. Platform and toolkit
Development platform, language, OS, testing frameworks, etc. Integration mechanisms
Web services platforms, enterprise search, etc. Risks
Security, Business, Cost
So, for Intranet:
* Mix of structured and unstructured info, non-spatial or minimally
* Non-spatial or minimally spatial requirements
* Suitable when info is restricted to authorized users/groups
* Small to mid-size (up to thousands of objects, not millions)
* LDAP authentication
* Archetypes, ArchGenXML
* ZODB storage unless special circumstances. Extra cost for RDB must be justified. Will support MySQL but not Oracle
Using RDB storage:
Good reasons: Legacy data that can't be moved, simultaneous access from other systems.
Bad reasons: Saying it's faster or more stable than ZODB. Making reports with Crystal, etc. Ad hoc SQL queries by power users (instead provide capability to periodically load data into RDB - much cheaper solution).
How fast can they make these apps?
15 days to produce apps with ZODB storage given UML diagram produced jointly by customer and developer with up to 5 content types, 1 custom view per type, 5 reports and/or custom search pages, standard editing pages, 6 custom permissions, 2 custom roles, 1 custom workflow.
Predictable cost requires:
* Use of shared product providing common elements: LDAP setup, provide application mixin class with common behavior, customize portal look/feel.
* Better for users as well as developers More coverage: ploneconf2007