Being a packaging supplier we are not utilizing the Walmart scorecard however we have extensive experience with the Walmart modeling software. I've found this to be very useful when comparing like materials and sometimes a bit challinging when comparing not-in-kind materials. For the not-in-kind comparisons Compass does a nice job providing the data and then you need to decide what that data means to you. We also have had good experience with Gabi when we used it to conduct an LCA comparing different paper grades used to manufacture corrugated. I think the key is deciding how you are going to use the tool and then select which tool(s) are right for you. We continue to utilize all three of the ones mentioned above.

From my experience working with and looking at the results from a wide variety of tools, I have developed several strong beliefs.

First is that without good data (material and process factors regarding energy use, GHG emissions, end of life impacts, etc.), it doesn't matter what tool you use; you'll be misled and will likely then mislead others to whom you communicate results. For me, two key elements about data are transparency and the ability to override default data when better or specific knowledge exists.

Transparency means clearly referenced sources showing when the data was generated and naming the sample size, geography, etc. This provides confidence and credibility regarding the base information on which an analysis is conducted, and also highlights shortcomings when data is old or from not very relevant industry segments. The more rigorous the tool, generally the better the data, but "caveat modeler" if you fail to test the assumptions upon which the supplied data is based. In particular, looking for biases built into the data that skew results towards the desired types of conclusions. Being able to override the default assumptions to test the impact of different inputs is crucial to knowing how important the accuracy of of a particular piece of data is, and also permits substituting what a user may know to be more accurate, up-to-date, or relevant data.

Few of the streamlined tools are good at fully documenting their sources, and often have out-of-date or geographically specific data that is not explained. The Wal-Mart scorecard and most other tools do not allow overriding default data, which is a big limitation. The SavvyPak tool, which I use regularly and also have a limited hand in refining and marketing (full disclosure!), is particularly good in this regard, offering the ability to substitute preferred data easily. It is then incumbent on the user to document the source of the data in the context of reporting the results, coming full circle to transparency. GaBi and SimaPro, two tools which have the ability to model very rigorously more aspects of environmental impact than most streamlined tools, use the EcoInvent database, which has decent documentation, and also permit inputting the specific data a user may prefer.

The second belief is about completeness. Just modeling raw materials, but ignoring converting process impacts and differences misses key pieces of the puzzle. The scorecard essentially ignores converting and COMPASS in many cases seems to lump together raw materials and converting, making deconstruction of the data to understand relative impacts difficult. GaBi, SimaPro and SavvyPack look at all aspects of creating a package and allow good customization of material and process combinations, broadening their applicability and making them more representative of real life. GaBi, SimaPro and COMPASS aim to account for many more impacts, including things like aquatic impacts and the like, although I am a bit concerned that in some of the more esoteric areas there is a great deal of scientific uncertainty today, and I worry about people putting too much stock in these results in comparing materials and package formats.

The third belief is avoiding undisclosed biases. Wal-Mart is explicit the scorecard is a business tool for their company, and their clarity about that is appropriate and admirable, as it does contain their biases about what is important. But other companies or individuals may have very different weighting schemes that incorporate different metrics, and they should be able to use what they want, so long as they are equally clear about how they are building those preferences into the analysis. When an analysis is reduced to a single number, one can almost always be sure there are biases built in and it should be a signal for ferreting them out to understand the implications.

Algorithms should be material and process neutral, and should be applied consistently to close the modeling loop - good data with sound and complete calculations combined to yield meaningful and actionable results. Again, GaBi, SimaPro and SavvyPack do a good job in this regard and are very flexible in handling many materials, converting process and package formats.

Unfortunately, there are not short or easy answers in comparing the available tools, as they vary greatly in intent, approach, flexibility, complexity and rigor. My advice is to become familiar with them and chose the one(s) that most closely match your desired outputs and your capabilities. BUT, don't scrimp on spending the time to develop a clear understanding of the strengths and limitations of whatever system(s) you chose to use, and be clear about those when you communicate results. LCA's are serious business and the temptation to turn the results into sound bites is strong; being open about data, completeness and bias puts the discussion back on solid ground.

I am currently doing a review of software resources, and have only a few preliminary comments:
1) Wal-mart’s score card, although it is based on lifecycle thinking is not really an LCA tool, and it only looks at climate change, as well as imbedding Wal-Mart’s value system, which the user may not share.
2) Sima-Pro and GaBi are the two leading LCA software resources globally, and are designed for LCA professionals they are flexible, comprehensive and transparent, but not something for the amateur.
3) Transparency is a big issue in all the specialized LCA software tools. Caveat emptor.
4) Data sources are also big issues… garbage in and garbage out.

What methodology, process and potentially software you use depends to some degree on what type of packaging you are looking to analyze. The Walmart Scorecard is not really meant to be an LCA tool and is a inflexible in its material choices. You cannot override or add materials. For analyzing some materials, the Walmart Scorecard would not be the best tool. Also, Compass does not take into account freight, if I recall correctly, and therefore you are missing a large piece of the puzzle.

As the others have written here transparency is critical. I am seeing so many tools that use a bit different weighting or methodology which is concerning because there are companies using this type of analysis to compare themselves to others, when the analysis process is different. Hence the need for complete transparency.

I think most would agree, that unless it's an LCA program, most scorecards are really more of a comparitive tools than anything else. Directionally, they will give you an idea of which package or material might be more environmentally beneficial than another candidate.

A tool is only as good as it's data, and that data is constantly changing due to the industry. So whether is the Wal-Mart Packaging Scorecard, COMPASS or anyone else's tool of choice, my advice would be to plug your data into 2 or 3 different tools and look at all the results to get a basic feel for your current or test package/ material.

If you intend on making an environmental claim with facts and figures behind it, a life cycle analysis might be the best route.
Although it will cost you some time and money to follow through with an LCA, it would be your best option to avoid green-washing.