Invalid NULL value error when upgrading from 2.4.8 to 3.1

Hi,

I am in the process of upgrading 14 of our ojs installations to 3.1.
I managed to get 3 of them done but on the 4th one I’m stuck with the following error.

ERROR: Upgrade failed: DB: Invalid use of NULL value

Here is the complete output of the CLI tool

[load: upgrade.xml]
[version: 3.1.0.1]
[code: Installer Installer::syncSeriesAssocType]
[data: dbscripts/xml/upgrade/2.4.0_preupdate_usageStatistics.xml (skipped)]
[data: dbscripts/xml/upgrade/2.4.0_preupdate_usageStatistics2.xml (skipped)]
[code: Installer Installer::removeReviewEntries]
[data: dbscripts/xml/upgrade/3.0.0_preupdate.xml]
[data: dbscripts/xml/upgrade/3.0.0_adaptTimedViews.xml (skipped)]
[data: dbscripts/xml/upgrade/3.0.0_adaptReferrals.xml]
[data: dbscripts/xml/upgrade/3.0.0_adaptBooksForReview.xml]
[data: dbscripts/xml/upgrade/remove_timed_views_bots.xml (skipped)]
[data: dbscripts/xml/upgrade/3.0.0_preupdate_commentsToEditor.xml]
[code: Installer Installer::concatenateIntoAbout]
PHP Warning:  file_put_contents(/var/www/THIRD_PARTY/SATNT/v3_installation/cache/fc-journalSettings-1.php): failed to open stream: Permission denied in /var/www/THIRD_PARTY/SATNT/v3_installation/lib/pkp/classes/cache/FileCache.inc.php on line 90
[code: Installer Installer::concatenateIntoMasthead]
[data: dbscripts/xml/upgrade/3.1.0_preupdate_review_assignments.xml]
[data: dbscripts/xml/upgrade/3.1.0_preupdate_notes.xml]
[data: dbscripts/xml/upgrade/3.1.0_preupdate_payments.xml]
[schema: lib/pkp/xml/schema/common.xml]
ERROR: Upgrade failed: DB: Invalid use of NULL value

This is fresh 2.4.8 installation that I created and imported XML files we received from the previous company that managed this journal.

I have tried this a couple of times, restoring the backup db before every go

Let me know if you require any other info please.

Regards

Hi @Nardusg,

There’s probably a data error in your OJS 2.x database. To identify it, I’d suggest running

php tools/dbXMLtoSQL.php -schema print lib/pkp/xml/schema/common.xml

…after you’ve attempted an upgrade and failed, but before you’ve restored the database from backup. This will generate a list of SQL statements. Try executing the list, and you should see one of them cause the same error message as you saw before:

Invalid use of NULL value

Post which of the SQL statements caused that problem and we can use that to pin down the problem data.

Regards,
Alec Smecher
Public Knowledge Project Team

Hi @asmecher

I ran all the queries returned from that command , it is basically a list of create statements and creating of indexes but none of them produce the error mentioned

This was the output.

[Err] 1050 - Table 'versions' already exists
[Err] CREATE TABLE versions (
major                    INTEGER NOT NULL DEFAULT 0,
minor                    INTEGER NOT NULL DEFAULT 0,
revision                 INTEGER NOT NULL DEFAULT 0,
build                    INTEGER NOT NULL DEFAULT 0,
date_installed           DATETIME NOT NULL,
current                  TINYINT NOT NULL DEFAULT 0,
product_type             VARCHAR(30),
product                  VARCHAR(30),
product_class_name       VARCHAR(80),
lazy_load                TINYINT NOT NULL DEFAULT 0,
sitewide                 TINYINT NOT NULL DEFAULT 0
);
[Err] 1061 - Duplicate key name 'versions_pkey'
[Err] ALTER TABLE versions ADD  UNIQUE INDEX versions_pkey  (product_type, product, major, minor, revision, build);
[Err] 1050 - Table 'site' already exists
[Err] CREATE TABLE site (
redirect                 BIGINT NOT NULL DEFAULT 0,
primary_locale           VARCHAR(14) NOT NULL,
min_password_length      TINYINT NOT NULL DEFAULT 6,
installed_locales        VARCHAR(255) NOT NULL DEFAULT 'en_US',
supported_locales        VARCHAR(255),
original_style_file_name VARCHAR(255)
);
[Err] 1050 - Table 'site_settings' already exists
[Err] CREATE TABLE site_settings (
setting_name             VARCHAR(255) NOT NULL,
locale                   VARCHAR(14) NOT NULL DEFAULT '',
setting_value            TEXT,
setting_type             VARCHAR(6) NOT NULL
);
[Err] 1061 - Duplicate key name 'site_settings_pkey'
[Err] ALTER TABLE site_settings ADD  UNIQUE INDEX site_settings_pkey  (setting_name, locale);
[Err] 1050 - Table 'auth_sources' already exists
[Err] CREATE TABLE auth_sources (
auth_id                  BIGINT NOT NULL AUTO_INCREMENT,
title                    VARCHAR(60) NOT NULL,
plugin                   VARCHAR(32) NOT NULL,
auth_default             TINYINT NOT NULL DEFAULT 0,
settings                 TEXT,
                 PRIMARY KEY (auth_id)
);
[Err] 1050 - Table 'users' already exists
[Err] CREATE TABLE users (
user_id                  BIGINT NOT NULL AUTO_INCREMENT,
username                 VARCHAR(32) NOT NULL,
password                 VARCHAR(255) NOT NULL,
salutation               VARCHAR(255),
first_name               VARCHAR(255) NOT NULL,
middle_name              VARCHAR(255),
last_name                VARCHAR(255) NOT NULL,
suffix                   VARCHAR(255),
gender                   VARCHAR(1),
initials                 VARCHAR(5),
email                    VARCHAR(255) NOT NULL,
url                      VARCHAR(2047),
phone                    VARCHAR(32),
mailing_address          VARCHAR(255),
billing_address          VARCHAR(255),
country                  VARCHAR(90),
locales                  VARCHAR(255),
date_last_email          DATETIME,
date_registered          DATETIME NOT NULL,
date_validated           DATETIME,
date_last_login          DATETIME NOT NULL,
must_change_password     TINYINT,
auth_id                  BIGINT,
auth_str                 VARCHAR(255),
disabled                 TINYINT NOT NULL DEFAULT 0,
disabled_reason          TEXT,
inline_help              TINYINT,
                 PRIMARY KEY (user_id)
);
[Err] 1050 - Table 'user_settings' already exists
[Err] CREATE TABLE user_settings (
user_id                  BIGINT NOT NULL,
locale                   VARCHAR(14) NOT NULL DEFAULT '',
setting_name             VARCHAR(255) NOT NULL,
assoc_type               BIGINT DEFAULT 0,
assoc_id                 BIGINT DEFAULT 0,
setting_value            TEXT,
setting_type             VARCHAR(6) NOT NULL
);
[Err] 1061 - Duplicate key name 'user_settings_user_id'
[Err] ALTER TABLE user_settings ADD  INDEX user_settings_user_id  (user_id);
[Err] 1061 - Duplicate key name 'user_settings_pkey'
[Err] ALTER TABLE user_settings ADD  UNIQUE INDEX user_settings_pkey  (user_id, locale, setting_name, assoc_type, assoc_id);
[Err] 1050 - Table 'sessions' already exists
[Err] CREATE TABLE sessions (
session_id               VARCHAR(128) NOT NULL,
user_id                  BIGINT,
ip_address               VARCHAR(39) NOT NULL,
user_agent               VARCHAR(255),
created                  BIGINT NOT NULL DEFAULT 0,
last_used                BIGINT NOT NULL DEFAULT 0,
remember                 TINYINT NOT NULL DEFAULT 0,
data                     TEXT,
domain                   VARCHAR(255)
);
[Err] 1061 - Duplicate key name 'sessions_user_id'
[Err] ALTER TABLE sessions ADD  INDEX sessions_user_id  (user_id);
[Err] 1061 - Duplicate key name 'sessions_pkey'
[Err] ALTER TABLE sessions ADD  UNIQUE INDEX sessions_pkey  (session_id);
[Err] 1050 - Table 'access_keys' already exists
[Err] CREATE TABLE access_keys (
access_key_id            BIGINT NOT NULL AUTO_INCREMENT,
context                  VARCHAR(40) NOT NULL,
key_hash                 VARCHAR(40) NOT NULL,
user_id                  BIGINT NOT NULL,
assoc_id                 BIGINT,
expiry_date              DATETIME NOT NULL,
                 PRIMARY KEY (access_key_id)
);
[Err] 1061 - Duplicate key name 'access_keys_hash'
[Err] ALTER TABLE access_keys ADD  INDEX access_keys_hash  (key_hash, user_id, context);
[Err] 1050 - Table 'notifications' already exists
[Err] CREATE TABLE notifications (
notification_id          BIGINT NOT NULL AUTO_INCREMENT,
context_id               BIGINT NOT NULL,
user_id                  BIGINT,
level                    BIGINT NOT NULL,
type                     BIGINT NOT NULL,
date_created             DATETIME NOT NULL,
date_read                DATETIME,
assoc_type               BIGINT,
assoc_id                 BIGINT,
                 PRIMARY KEY (notification_id)
);
[Err] 1061 - Duplicate key name 'notifications_context_id_user_id'
[Err] ALTER TABLE notifications ADD  INDEX notifications_context_id_user_id  (context_id, user_id, level);
[Err] 1061 - Duplicate key name 'notifications_context_id'
[Err] ALTER TABLE notifications ADD  INDEX notifications_context_id  (context_id, level);
[Err] 1061 - Duplicate key name 'notifications_assoc'
[Err] ALTER TABLE notifications ADD  INDEX notifications_assoc  (assoc_type, assoc_id);
[Err] 1050 - Table 'notification_settings' already exists
[Err] CREATE TABLE notification_settings (
notification_id          BIGINT NOT NULL,
locale                   VARCHAR(14),
setting_name             VARCHAR(64) NOT NULL,
setting_value            TEXT,
setting_type             VARCHAR(6) NOT NULL
);
[Err] 1061 - Duplicate key name 'notification_settings_pkey'
[Err] ALTER TABLE notification_settings ADD  UNIQUE INDEX notification_settings_pkey  (notification_id, locale, setting_name);
[Err] 1050 - Table 'notification_subscription_settings' already exists
[Err] CREATE TABLE notification_subscription_settings (
setting_id               BIGINT NOT NULL AUTO_INCREMENT,
setting_name             VARCHAR(64) NOT NULL,
setting_value            TEXT,
user_id                  BIGINT NOT NULL,
context                  BIGINT NOT NULL,
setting_type             VARCHAR(6) NOT NULL,
                 PRIMARY KEY (setting_id)
);
[Err] 1050 - Table 'notification_mail_list' already exists
[Err] CREATE TABLE notification_mail_list (
notification_mail_list_id BIGINT NOT NULL AUTO_INCREMENT,
email                    VARCHAR(90) NOT NULL,
confirmed                TINYINT NOT NULL DEFAULT 0,
token                    VARCHAR(40) NOT NULL,
context                  BIGINT NOT NULL,
                 PRIMARY KEY (notification_mail_list_id)
);
[Err] 1061 - Duplicate key name 'notification_mail_list_email_context'
[Err] ALTER TABLE notification_mail_list ADD  UNIQUE INDEX notification_mail_list_email_context  (email, context);
[Err] 1050 - Table 'email_templates_default' already exists
[Err] CREATE TABLE email_templates_default (
email_id                 BIGINT NOT NULL AUTO_INCREMENT,
email_key                VARCHAR(64) NOT NULL,
can_disable              TINYINT NOT NULL DEFAULT 1,
can_edit                 TINYINT NOT NULL DEFAULT 1,
from_role_id             BIGINT,
to_role_id               BIGINT,
                 PRIMARY KEY (email_id)
);
[Err] 1061 - Duplicate key name 'email_templates_default_email_key'
[Err] ALTER TABLE email_templates_default ADD  INDEX email_templates_default_email_key  (email_key);
[Err] 1050 - Table 'email_templates_default_data' already exists
[Err] CREATE TABLE email_templates_default_data (
email_key                VARCHAR(64) NOT NULL,
locale                   VARCHAR(14) NOT NULL DEFAULT 'en_US',
subject                  VARCHAR(120) NOT NULL,
body                     TEXT,
description              TEXT
);
[Err] 1061 - Duplicate key name 'email_templates_default_data_pkey'
[Err] ALTER TABLE email_templates_default_data ADD  UNIQUE INDEX email_templates_default_data_pkey  (email_key, locale);
[Err] 1050 - Table 'email_templates' already exists
[Err] CREATE TABLE email_templates (
email_id                 BIGINT NOT NULL AUTO_INCREMENT,
email_key                VARCHAR(64) NOT NULL,
assoc_type               BIGINT DEFAULT 0,
assoc_id                 BIGINT DEFAULT 0,
enabled                  TINYINT NOT NULL DEFAULT 1,
                 PRIMARY KEY (email_id)
);
[Err] 1061 - Duplicate key name 'email_templates_email_key'
[Err] ALTER TABLE email_templates ADD  UNIQUE INDEX email_templates_email_key  (email_key, assoc_type, assoc_id);
[Err] 1061 - Duplicate key name 'email_templates_assoc'
[Err] ALTER TABLE email_templates ADD  INDEX email_templates_assoc  (assoc_type, assoc_id);
[Err] 1050 - Table 'email_templates_data' already exists
[Err] CREATE TABLE email_templates_data (
email_key                VARCHAR(64) NOT NULL,
locale                   VARCHAR(14) NOT NULL DEFAULT 'en_US',
assoc_type               BIGINT DEFAULT 0,
assoc_id                 BIGINT DEFAULT 0,
subject                  VARCHAR(120) NOT NULL,
body                     TEXT
);
[Err] 1061 - Duplicate key name 'email_templates_data_pkey'
[Err] ALTER TABLE email_templates_data ADD  UNIQUE INDEX email_templates_data_pkey  (email_key, locale, assoc_type, assoc_id);
[Err] 1050 - Table 'oai_resumption_tokens' already exists
[Err] CREATE TABLE oai_resumption_tokens (
token                    VARCHAR(32) NOT NULL,
expire                   BIGINT NOT NULL,
record_offset            INTEGER NOT NULL,
params                   TEXT
);
[Err] 1061 - Duplicate key name 'oai_resumption_tokens_pkey'
[Err] ALTER TABLE oai_resumption_tokens ADD  UNIQUE INDEX oai_resumption_tokens_pkey  (token);
[Err] 1050 - Table 'plugin_settings' already exists
[Err] CREATE TABLE plugin_settings (
plugin_name              VARCHAR(80) NOT NULL,
context_id               BIGINT NOT NULL,
setting_name             VARCHAR(80) NOT NULL,
setting_value            TEXT,
setting_type             VARCHAR(6) NOT NULL
);
[Err] 1061 - Duplicate key name 'plugin_settings_plugin_name'
[Err] ALTER TABLE plugin_settings ADD  INDEX plugin_settings_plugin_name  (plugin_name);
[Err] 1061 - Duplicate key name 'plugin_settings_pkey'
[Err] ALTER TABLE plugin_settings ADD  UNIQUE INDEX plugin_settings_pkey  (plugin_name, context_id, setting_name);
[Msg] Finished - Unsuccessfully

Could it be in the file that runs after common.xml?

Hi @Nardusg,

Sorry, I got the command slightly wrong. Try:

 php tools/dbXMLtoSQL.php -schema print_upgrade lib/pkp/xml/schema/common.xml

Regards,
Alec Smecher
Public Knowledge Project Team

Hi @asmecher

Awesome thank you very much. So this is the query giving the error.

ALTER TABLE users MODIFY COLUMN first_name VARCHAR(255) NOT NULL;
[Err] 1138 - Invalid use of NULL value

Makes sense, I remember in the imports I had issues with this as well.

I should be able to fix this from here.

Thank you.