UNANSWERED QUESTIONS cast more doubt…
The memorandum below was sent to all the people involved in crafting and promoting the CSI. It was sent electronically and via certified mail. No response was received.
To see the complete correspondence to CSI click here.
To: Free Cruising Guides; Marina Zar-Par; Frank Virgintino, Author and Founder; Catherine Hebson, PhD, Director; Nathalie Virgintino, PhD, Editor; Amaia Agirre, Researcher and Author
From: Mary Stone, Captain, M/V Ms Astor (firstname.lastname@example.org)
Subject: Questions regarding The Caribbean Security Index (CSI) to understand why it is not pseudo-science, false, deceptive and misleading.
Previous requests to answer simple questions to Dr. Hebson have gone unanswered. The two questions were:
- What is meant by the comment “CSI is at best a well crafted probability analysis”?
- Have proper validation studies been carried out on CSI?
Since no answer has been forthcoming the conclusion is that CSI is neither “well crafted” nor valid. CSI has not defended its validity or how it was derived.
The addressees of this memorandum therefore have made false, deceptive and misleading claims regarding the efficacy of the Caribbean Security Index (CSI).
Consequently, it raises a myriad of other questions which follow:
- Have proper validation studies been carried out to validate the CSI probabilities of crime against yachts in each indexed location, so that it is clear that the probabilities of crime against yachts do indeed correlate sufficiently well with the various criteria used as key indicators (or criteria such as Socio Economic data, unemployment and crime rates reported for each country)?
- It is stated that “CSI ratings are derived from an examination of the relationship between crime and socio-demographic factors such as unemployment, education level, and literacy; as well as the presence (or absence) of security, infrastructure, and history of crime in a country/island.” Do you use only general country crime statistics or reported crime against yachts to correlate with the other criteria and product the index?
- How do the general country crime rates reported relate to crimes against yachts at the indexed locations, and to what extent are they correlated with the broad economic data and used in the CSI?
- What are the sources for the data you use in the CSI and how current are they; specifically “… crime and socio-demographic factors such as unemployment, education level, and literacy; as well as the presence (or absence) of security, infrastructure, and history of crime in a country/island.”?
- What are the sources of the crime reports against yachts and how much does each source contribute to your overall report data?
- Why do you not provide attribution to your source data? How many reported crimes are recorded by you and how many from other sources? What is the mix of sources and how do you use the mixed data?
- To what extent do you calculate any ratios of crime against yachts relative to the number and type of yachts in a given anchorage or country over time?
- To what extent does the index use current year economic data to make probability estimates of crime against yachts for the current year?
- Does the index use country’s current year reported crime to make probability estimates of crime against yachts for the current year?
- Does the index use the country’s current year unemployment rates to make probability estimates of crime against yacht for the current year?
- Why is there very little differentiation in the index value for the countries where the index is calculated? (For example the low index is 5.3 and the high is 9.8 on a 10 point scale which suggests that almost all countries have an index above the median of the scale). How can that be?
- Why are there so many countries with an index of N/A?
- Does the trend arrow use a quantitative trend analysis to determine its direction or is it just a considered opinion of the author(s)?
- How is the color code for each country derived and what do the colors mean?
- Why is the Grenada Trinidad passage coded red with an index of 6.0 and Haiti is coded yellow with an index of 5.6 in harbors (and Haiti has had violent assaults more so than the Trinidad Grenada Passage)?
- How do you calculate an index of 6.0 for the Grenada Trinidad passage (which is not a country) using “an examination of the relationship between crime and socio-demographic factors such as unemployment, education level, and literacy; as well as the presence (or absence) of security, infrastructure, and history of crime in a country/island.”?
- Using “an examination of the relationship between crime and socio-demographic factors such as unemployment, education level, and literacy; as well as the presence (or absence) of security, infrastructure, and history of crime in a country/island”, how can you calculate the CSI index for specific locations in marinas, harbors or anchored out when there is only very general country data available at a country/island level and no data available to the specific marina, harbor or anchorage?
- To what extent are the discussions of mitigating factors based on quantitative data used to calculate the CSI?
- Why do you divide the Caribbean into quadrants yet do not have a CSI for the quadrant (only for countries, islands, anchorages, marinas and moorings)?
- The 2016 CSI report said “The Index is presented on a scale of 1 to 10 with notation. We can assess only reported incidents, a subset of all crimes against cruisers. There is little consistency in crime reporting and/or official recording either within one country or across all countries in the Caribbean.”, so how can you calculate a CSI on a 10 point scale when there is little consistency in reported data and you criticize the sources you claim to use?
- In some places you indicate that reported incidents by cruisers are not included in the data used to calculate the CSI. In other places you indicate you can only use reported data to calculate CSI. Can you clarify exactly where and how us use reported data?
- Why is there no guidance to cruisers on how to use the CSI 10 point scale to indicate what part of the scale related to high, medium and low risk?
- The places that score a lower index are well known trouble spots and an index that identifies them as such is non value added; so why do you even bother to calculate an index that resides in the mid to upper half of the scale for well-known problem spots?
- Is it accurate to assume that a CSI of 1, 2, or 3, would be considered high risk? And would a CSI of 4, 5 or 6 be considered moderately risky? And would a CSI of 7, 8, or 9 be considered low risk? How is it useful to cruisers to see that such an extreme number of locations have indices in the high end of the scale?
- The 2016 CSI report indicates “… that the majority of cruisers to the Caribbean start their journey from the east coast of North America…”. Can you cite your source for this claim given that the ARC rallies and others account for hundreds of boats arriving to the Caribbean every year from other locations?
- The 2016 CSI report indicates that “…, many cruise through the Bahamas and so CSI keeps tabs on the Bahamas. “, yet there is no information in the report on the Bahamas – why?
- In general the CSI clearly minimizes crime against yachts throughout the Caribbean. Of the 77 CSI ratings only 7 are below 7 and none below 5 – and all of this in the moderate risk part of the CSI scale. The 66 others have a CSI of over 7 indicating a LOW risk. Except for Venezuela which has CSI of from 4 to 5.4 (moderate) and yet there is no crime data to support any of this this year. How should a captain use this information in their risk analysis when there is so little differentiation?
- Why are there 15 locations that have N/A ?
- To what extent is CSI a commercial product designed to show the Caribbean as having very little risk to cruisers in support of the business interests represented by Free Cruising Guides, Marina Zar Par and the advertisers that contribute to the profits of the aforementioned so as to lure tourism money to the Caribbean at the expense of cruiser safety and security?
“The Caribbean Security Index (CSI) is a a tool to assist cruisers in assessing the probability of crime at ports and anchorages throughout the Caribbean. The CSI provides a means of assessing risk in a given area. The index is stated on a numerical scale (1-10) with mitigating factors included.”
“The index is derived from the examination of the relationship between crime as reported (from a variety of sources), unemployment, levels of education and literacy. Additional factors include average income, distribution of income, presence of security, infrastructure and past record and history of crimes in a given area.”
” … Ratings are for each country on a scale of 1-10 (10 safest) based on the combination of the factors mentioned above…”
Marina Zarpar provides the CARIBBEAN SECURITY INDEX. This index is published at the site and can be downloaded free. Unlike police type blotters which report crime by date, the CSI provides detailed probability analysis that can be used by the cruiser to determine the probability of various crimes against cruisers in different areas of the Caribbean.
This is false and deceptive advertising.
Since CSI has not been validated to prove that the published probabilities are any better than chance, the advertised claims that the CSI helps determine the likelihood of crime occurring in a given location is therefore false.
The CSI provides no proof its probabilities are validated, better than chance, or value added for a captain to use when attempting to assess risk of crime targeted at vessel and crew in a specific cruising location in the Caribbean. Therefore, CSI advertised claims that its probabilities are useful for assessing risk of crimes against yachts in the Caribbean are false. Moreover, the use of an improperly validated index mixed with liberal doses of biased opinion to indicate that locations are safe is misleading and possibly dangerous to unsuspecting captains who rely on the information to be true.
The CSI is advertised to be based on properly applied scientific, statistical, well-crafted risk analysis. This claim is false, deceptive and misleading. The author provides no proof of validity and no defense of CSI claims. The claims made by the CSI are unsubstantiated.
CSI appears to be a pseudoscience product designed to minimize concerns about crime against yachts in order to increase tourism spend to maximize profits for the principals, publishers and advertisers.
To date, CSI will not confirm that the proper statistical methodologies (specifically multiple regression analyses) have been properly applied nor how such related methodologies were applied even when a nondisclosure agreement was offered.
The Caribbean Security Index (CSI) should be retracted from any present and past mentions on freecruisingguides.com , http://marinazarpar.com/ and from any promotion by Marina Zar-Par or Free Cruising Guides in any media which currently advertise or mentions CSI. Do not publish or advertise CSI in the future in any media. Advertisers accepting adverts from Marina Zar_Par or FreeCruisingGuides.com should stop CSI related advertising immediately.
While the actual basis for such action is that the CSI not been shown to be valid and is deceptive and misleading, the publicly plausible reason could be that the source data is no longer of sufficient quality, quantity or reliability to be able to provide properly calculated probabilities.
Whatever the reason given, the CSI should be retired immediately because it is not valid, only false, deceptive and misleading.
In the absence of any validity confirmation and/or substantiative answers to the above questions, formal complaints of false, deceptive and misleading advertising have been prepared and will be released by 1 April 2017 to appropriate agencies and advertisers. If any action is taken by those agencies and advertisers, you will be notified by those entities according to their processes and procedures.
Below is a copy oF the CSI Report for your reference.
CAUTION – THIS REPORT HAS NOT BEEN VALIDATED
The questions above apply to this report:
PLEASE BE PATIENT – THIS WILL TAKE A MOMENT TO LOAD – It’s a .pdf file.csi_update 2016_12_31_0