For example, I once had a research prompt to find performance monitoring and testing tools for my previous .NET teams and found out the following challenges.
- IT Operations "testing" focuses on bandwidth, packets sniffing, and a more appliance approach. They want to get insight on the stability of systems.
- Developers are focused on personal toolkits of sorts. They can purchase such tools almost out of pocket and are easily approved through a PO, signed by a manager. Think "shrinkwrap" like the options from RedGate.
Rehman Zafar describes at least ten types of testing and I'm not an expert at them, but we did cover them in college (University for my European readers):
- Unit Testing
- Integration Testing
- Functional Testing
- System Testing
- Stress Testing
- Performance Testing
- Usability Testing
- Acceptance Testing
- Regression Testing
- Beta Testing
Content Delivery Testing is "Easy"The good news is that all of your Content Delivery testing has the same "footprint" of what you've tested before. You'll still test markup, script, and application code after introducing Tridion as your CMS. This covers delivery-side Functional, System, Stress, Performance, Usability, Acceptance, Regression, and visitor Beta Testing under the Nunoism, "How would you do it without Tridion?" The only thing somewhat unique to Tridion is Acceptance Testing might be on a specific server that Tridion publishes to and that any issues surfaced might relate to Tridion delivery-side features.
What's left includes testing things like the Content Manager, its Editors, and "Publishing."
Templating Testing ExamplesBefore I get into all the things you might consider on the Content Manager (CM) side, here are some unit and integration test ideas followed some recommendations for Functional Testing.
- Do Unit Testing with Template Builder, optionally using the examples from T-Cubed
- An automated integration testing post and though not quite Integration Testing, but Building Blocks describes a Continuous Integration approach for C# Template Building Blocks
Aside from template code, which is probably the best quick win you might start with, you can break CM-side testing down along different types of testing as well as by interface.
On the Content Manager side, your focus for testing should be on the implementation or the specific Tridion content model, extensions, workflow, and automation developed for an organization. Testing Tridion itself is really up to the vendor, though you should keep notes and report things that feel like a product issue rather than a implementation problem.
- Functional Testing should be based on your functional requirements (learn more about an SDL Tridion Functional Design)
- Usability Testing falls a lot on the product itself, but whether the entry forms (based on schema) are "usable" depends on how you've created your Tridion content model (100 fields for a promo isn't Tridion's fault, but that fact that you added 100 fields to the promo schema)
- Beta Testing and Acceptance Testing for new CM-side functionality would happen in a lower Tridion DTAP environment
- Regression Testing would also be based on your functional requirements to an extent
- Organization and Authorization: Blueprint, Folders, Structure Groups (Pages), and Category/Keywords
- Creation in Content Manager Explorer: schemas, pages (add components with templates), and field options
- Creation in Experience Manager: page types, content types, and regions, as well as publishing to just Staging and "Update Preview
- Publishing and Integrations: publishing pages to the site(s) and confirming the results, you can check this down to separate phases (rendering, transport, deployment, and commitment) if needed
- Testing across each publication targets and CMS environments
For Experience Manager, it really depends on your implementation. But you'll want to confirm functionality in terms of:
- Page Types and Content/Component Types
- Items saved to correct publication
- Visibility and permissions if set up
- Confirm Component Presentation combinations work as expected (available schemas work with different templates), prioritizing important combinations
- Test regions work as expected for allowed content types (pairs of components and component templates)
Final (Initial) ThoughtsHere are some other considerations, recommendations and gotchas. Focus on the parts that are important to you and/or combining scenarios into your test criteria.
- Scope and tools
- Break down tests down by important variables (browser, schema, field, template, extension, steps in the process), list out options (in Excel or a similar list), and test manually.
- Optionally consider a tool like Selenium for screen automation and/or log analysis / reporting tools (BareTail, Splunk, etc). I haven't had a chance at using these specifically with the new UI, but it can they can help you validate the tags and rendered pages in general.
- Implementation option: for ease-of-testing, add template and component information via TBB, only rendered to Staging. These could let you manually inspect and validate tcm ids and other information.
- Consider using validation tools or sites to confirm markup
- Test with at least the browser your authors will use ("it works in <my browser>" discussions occasionally apply to the CME)
- Test rich text fields
- See if empty fields affect your presentation / editability
- Confirm required fields match your design
- Verify any schema facets (xsd filters)
- Copy and paste text, use shortcut combinations, and otherwise replicate what your authors would expect from the editor
- Character combinations and special characters (&, -, -, (c), (r))-do templates and presentation escape or encode these as expected
- Confirm UTF-8 character set (or evaluate why you'd choose something different!)
- Verify XSLT rich text filter behavior on fields (if any)
- Evaluate rendering
- Check for issues with any code-driven rendering functionality
- Validate the non-rendered source (from CD file system, if possible)
- Experience Manager:
- Tip: avoid block elements such as headings or <div> within <span>
- Browsers may force changes to bad markup, affecting Experience Manager
Consider creating and using a set of dedicated test content which could help with future (regression) testing. Capture any of the typical issues you've seen with CMS and Web technology in general. For future schema, consider collecting some test text that includes the above test scenarios.But this is just an overview (that probably would read better as a video or checklist) and it isn't necessarily a new topic (read Dominic Cronin's Total Quality post circa 2006). Got thoughts on "Testing Tridion?" Tell us about them in a comment or a post. If you've been trying to keep up with the new new community, we're almost through Q1 and "testing" might be a great topic for your next post or two.