Bot list for the counter metric

carola · March 1, 2017, 7:36am

Hi,

I am referring to this thread in this forum.

beghelli wrote:

Actually OJS uses the COUNTER project bots list to avoid bot access.

My questions:

Is it this list: https://www.projectcounter.org/code-of-practice/appendices/850-2/ ?
Are the items of the list automatically included into the Usage Statistics Plugin or do you include them manually? And if so: how often do you update the Plugin?

Thanks,
Carola

ctgraham · March 1, 2017, 1:38pm

The bot list was taken from the COUNTER Code of Practice. In OJS 3.0.2, it is a snapshot from January 2011.

A later version is available:
https://www.projectcounter.org/wp-content/uploads/2016/10/COUNTER_Robots_list_Oct-2016.txt

The code will look for any one file in the directory where the current file lives:

github.com

pkp/pkp-lib/blob/ojs-stable-3_0_2/plugins/generic/usageStats/PKPUsageStatsLoader.inc.php#L570-L588


	/**
	 * Get the COUNTER robot list file.
	 * @return mixed string or false in case of error.
	 */
	private function _getCounterRobotListFile() {
		$file = null;
		$dir = PKP_LIB_PATH . DIRECTORY_SEPARATOR . $this->_plugin->getPluginPath() . DIRECTORY_SEPARATOR . 'lib' . DIRECTORY_SEPARATOR . 'counter';

		// We only expect one file inside the directory.
		$fileCount = 0;
		foreach (glob($dir . DIRECTORY_SEPARATOR . "*") as $file) {
			$fileCount++;
		}
		if (!$file || $fileCount !== 1) {
			return false;
		}

		return $file;
	}

This would make for a good pull request.

aguen · March 1, 2017, 3:51pm

Hi all,

A bot list from January 2011 sounds really outdated! I think, if “COUNTER complience” shall be taken seriously the current COUNTER bot list should be used. However, more timely updates of the bot list might cause problems for internal consistencies of statistics so perhaps a configurable solution might be best (letting users choose to update to the current bot list or not).

carola · March 3, 2017, 10:59am

Hi @ctgraham, @aguen

Thank you for the information!

I agree with aguen that major updates after such a long time period may create inconsistencies in the data.

I will first check what the difference would be. Then I will think about how we want to deal with that topic in the future.

best,
Carola