Queued payments

OJS 3.3.0.14

People who have ordered article reprints called asking for receipts. Their article order was stored in payments page but no receipt was sent from paypal. Looked at paypal and no record of the order. Customer got reprint…but was not charged. Looked at queue_payments and there are many queued payments in there including the order from this particular customer.

Seems that this functionality is broken or something is stopping this. We do get occasional orders in from paypal.
How do we debug this?

Thanks!

@asmecher

More details:

Briefly, we have determined that there are many transactions that are the queued_payments table that have NOT been charged/processed properly at paypal. In our case the majority are reprint purchases but there may be some subscription purchases.

The paypal module from OJS and/or the base code for OJS seems to have a bug that causes the transaction to fail. Or it is possible that paypal is dropping the charge somehow. OJS opens access to the content but payment was never received. We found out when a customer asked for a receipt but we could not find the charge on paypal.

What makes this sketchy is that it does not fail all the time. If we run tests, it works. But there is something in there that causes the charge to fail about 60% of the time!

Have you seen this issue and what can you share with me? Finally, I am trying to debug through the logs and have requested the transactions logs from paypal. Need to find the code that handles this and find out how it works. Probably need help of a good PHP programmer for a couple of hours. This is a big crisis for us right now.

Thanks, radjr

Hi @radjr,

I think you should probably be looking in completed_payments rather than queued_payments. It’s fine for records to hang around queued_payments, and they will neither grant or deny access. It’s completed_payments that is used to look up whether a payment condition has been fulfilled.

I would suggest debugging access that seems to be unwarranted by looking through the view function in pages/article/ArticleHandler.inc.php. There are many ways that access can be granted or denied, and using e.g. error_log calls to make it clear which are applying in a case you’re trying to debug will help narrow down what is happening.

Cross-linking: User gains access to article even though payment does not complete. Transactions that are in the queued_payments table are not charged/processed properly at paypal · Issue #9735 · pkp/pkp-lib · GitHub

Regards,
Alec Smecher
Public Knowledge Project Team

I think the main point is being missed. Payments are being queued and the “Post” that is sent to paypal is broken.(see details below) The user is granted access to the reprint even though payment was not completed.

Please tell us where the “post” to paypal is constructed so we can determine why the post is broken.

Successful Transaction:

Request Path:
POST /v1/payments/payment/PAYID-MXF32ZQ56X69216U37604225/execute

Failed Transactions:

POST /v1/payments/payment

Hi @radjr,

The need for payment is communicated to PayPal in PaypalPaymentForm.inc.php in the display function. (This uses the 3rd-party Omnipay library.)

PayPal then contacts OJS via PaypalPaymentPlugin::handle to communicate fulfillment of the payment, which converts the queued payment into a completed payment.

Regards,
Alec Smecher
Public Knowledge Project Team

Thanks!

What is the possibility that an unserialize PHP error is causing the error?

[Fri Feb 16 13:29:56.578989 2024] [proxy_fcgi:error] [pid 9202:tid 140659499296512] [remote 157.55.39.61:2592] AH01071: Got error ‘PHP message: PHP Notice: unserialize(): Error at offset 81 of 251 bytes in [redacted]l/ojs/lib/pkp/classes/db/DAO.inc.php on line 254PHP message: PHP Notice: unserialize(): Error at offset 81 of 251 bytes in [redacted]/public_html/ojs/lib/pkp/classes/db/DAO.inc.php on line 254’

How can I tell what version of Omnipay is included in OJS 3.3?

Thanks, radjr

Seems that if a potential customer abandons the order, the order stays in the queue_payment table. Can you confirm that please?
Thanks!

Hi @radjr,

What is the possibility that an unserialize PHP error is causing the error?

… PHP Notice: unserialize(): Error at offset 81 of 251 bytes in …

It’s possible. OJS stores queued payment data using the PHP serialize / unserialize functions; if OJS can’t retrieve and unserialize a queued payment, then it’s not going to be able to complete it. The commonest cause of these unserialize errors is if you change your database character set configuration without converting the data that’s stored in the table.

How can I tell what version of Omnipay is included in OJS 3.3?

Omnipay is included as a Composer dependency. See the Composer lockfile for the specific versions that are shipped: plugins/paymethod/paypal/composer.lock

Seems that if a potential customer abandons the order, the order stays in the queue_payment table. Can you confirm that please?

Yes, that’s correct.

Regards,
Alec Smecher
Public Knowledge Project Team

Here is the data from the composer.lock file. This shows version 3.0.4. We know from Paypal that this is still version 1 as they call it and it is deprecated many years ago.

“name”: “omnipay/common”,
“version”: “v3.0.4”,
“source”: {
“type”: “git”,
“url”: “GitHub - thephpleague/omnipay-common: Core components for the Omnipay PHP payment processing library”,
“reference”: “d6a1bed63cae270da32b2171fe31f820d334d452”
},
“dist”: {
“type”: “zip”,
“url”: “https://api.github.com/repos/thephpleague/omnipay-common/zipball/d6a1bed63cae270da32b2171fe31f820d334d452”,
“reference”: “d6a1bed63cae270da32b2171fe31f820d334d452”,
“shasum”: “”
},
“require”: {
“moneyphp/money”: “^3.1”,
“php”: “^5.6|^7”,
“php-http/client-implementation”: “^1”,
“php-http/discovery”: “^1.2.1”,
“php-http/message”: “^1.5”,
“symfony/http-foundation”: “^2.1|^3|^4|^5”
},
“require-dev”: {
“omnipay/tests”: “^3”,
“php-http/mock-client”: “^1”,
“phpro/grumphp”: “^0.14”,
“squizlabs/php_codesniffer”: “^3.5”
},
“suggest”: {
“league/omnipay”: “The default Omnipay package provides a default HTTP Adapter.”
},
“type”: “library”,
“extra”: {
“branch-alias”: {
“dev-master”: “3.0.x-dev”
}
},
“autoload”: {
“psr-4”: {
“Omnipay\Common\”: “src/Common”
},
“classmap”: [
“src/Omnipay.php”

Thank you.

RE: if you change your database character set configuration without converting the data that’s stored in the table.

The only time a table would have been changed would have been on a version upgrade. Do you remember when the ojs upgrade required a change in type? Also for a newbie, how would I recovert the data? Again many thanks.

Message from emails that were posted back to the PKP forum.

Can you tell me where the paypal “post” requests are being constructed and sent to Paypal. I want to see what is being used to construct the link and see if there is a way they can be logged. Thanks, radjr

The current version of omnipay is 3.2.1 with support for newer versions of guzzle, etc.

It is unclear from their change log if any other bugs have been fixed between 3.0.4 and 3.2.1 that may affect the bug being reported above.

** Is there a script available to reconcile queued_payments, completed_payments and access to subscriptions or reprints? The goal is to find those users who did not pay(1) and still have access.
(1) We have proven that is the payment gateway fails in a certain way, the user still gains access to the item they were trying to purchase.

Details below.

v3.2 - 2021-06-01

Omnipay 3.2 is compatible with PHP8. This is done by upgrading the test suite to PHPUnit 8/9, with the release of omnipay/tests v4 and omnipay/common v3.1. This change is primarily for gateway developers, to make it possible to actually test PHP8, but they will need to upgrade their tests to use PHPUnit 9 (the currently supported PHPUnit version).

v3.1 - 2020-10-29

Omnipay 3.1 uses Guzzle 7 by default (using the Guzzle 7 adapter). This doesn’t change omnipay-common because they will work with any compatible Http Client. The minimum PHP versions is bumped to 7.2 because of this.

So in the ArticleHandler.php, it looks in the completepayment table to get access to the files via the completedpayment call. BUT, see the code below. The code suggests that if the transaction just took place, you just give them access. Can you share the thinking? Is there a race condition?

/* if the article has been paid for then forget about everything else
* and just let them access the article /
$completedPaymentDao = DAORegistry::getDAO(‘OJSCompletedPaymentDAO’); /
* @var OJSCompletedPaymentDAO $completedPaymentDao */
$dateEndMembership = $user->getData(‘dateEndMembership’, 0);
if ($completedPaymentDao->hasPaidPurchaseArticle($userId, $submission->getId())
|| (!is_null($dateEndMembership) && $dateEndMembership > time())) {
$this->issue = $issue;
$this->article = $submission;
return true;
} elseif ($paymentManager->purchaseArticleEnabled()) {
$queuedPayment = $paymentManager->createQueuedPayment($request, OJSPaymentManager::PAYMENT_TYPE_PURCHASE_ARTICLE, $user->getId(), $submission->getId(), $context->getData(‘purchaseArticleFee’));
$paymentManager->queuePayment($queuedPayment);

                        $paymentForm = $paymentManager->getPaymentForm($queuedPayment);
                        $paymentForm->display($request);
                        exit;
                    }

Hi @radjr,

Here is the data from the composer.lock file. This shows version 3.0.4. We know from Paypal that this is still version 1 as they call it and it is deprecated many years ago.

I suspect you’re referring to Omnipay’s PayPal adapter making use of the PayPal v1 API. I don’t have any particular insight into this, but there appears to be a longstanding issue filed on the PayPal adapter’s Github repo for this. I don’t see any concrete plans for the V1 API being retired e.g. by a specific date; hopefully if/when that happens, Omnipay will update their integration. Meanwhile it should be working.

RE: if you change your database character set configuration without converting the data that’s stored in the table.

The only time a table would have been changed would have been on a version upgrade. Do you remember when the ojs upgrade required a change in type? Also for a newbie, how would I recovert the data? Again many thanks.

This is inherently a fussy collation/DBA question, so unfortuantely it’s not newbie friendly. Long story short, OJS faithfully connects with whatever character set you specify in config.inc.php, but over the years MySQL has changed its default character set, and it’s possible to accidentally garble things with MySQL dump/reload cycles if you don’t specify the character set on the command line, which has sometimes happened to OJS users. The OJS serialize/unserialize functions are picky about string lengths, and a UTF-8 string that’s been incorrectly encoded in the database may trigger that. We don’t use serialize/unserialize for much and it’s generally not critical. That may be what’s happened here.

If I were approaching this from your perspective, and not wanting to get lost in details, I’d try to capture a little more information about the warnings you’re seeing to know whether they’re important or not. Go to the line of code that’s giving you the warning, and modify it to capture details in your error log when the content can’t be unserialized.

Is there a script available to reconcile queued_payments, completed_payments and access to subscriptions or reprints? The goal is to find those users who did not pay(1) and still have access.
(1) We have proven that is the payment gateway fails in a certain way, the user still gains access to the item they were trying to purchase.

No, there isn’t a script to reconcile queued payments with completed payments. When a queued payment is fulfilled via confirmation from PayPal, it is removed from queued_payments, and a completed_payments entry is created instead. Only completed_payments is checked to see whether a user should have access to paywalled contents; whether or not there is a queued_payment entry for that user is irrelevant.

So in the ArticleHandler.php, it looks in the completepayment table to get access to the files via the completedpayment call. BUT, see the code below. The code suggests that if the transaction just took place, you just give them access. Can you share the thinking? Is there a race condition?

If you use the code formatting tools available in this forum software it’ll be easier to read; I’m not sure if you’re highlighting a specific line of code. But no, I don’t think there’s a race condition. PayPal should fulfill the payment in OJS before redirecting the user back to the article webpage.

Just a heads-up that I can only give limited help with these subjects, particularly when they require server-side investigation; it can be incredibly time-consuming to intuit what’s happening.

Regards,
Alec Smecher
Public Knowledge Project Team

Thanks.

“I suspect you’re referring to Omnipay’s PayPal adapter making use of the PayPal v1 API. I don’t have any particular insight into this, but there appears to be a longstanding issue filed on the PayPal adapter’s Github repo for this. I don’t see any concrete plans for the V1 API being retired e.g. by a specific date; hopefully if/when that happens, Omnipay will update their integration. Meanwhile it should be working.”

We have had several people comment that they will not use paypal as it looks “retail” in nature versus professional. Additionally, paypal “pushes” people to sign in and many don’t like that, and finally, the option to use a credit card is down on the paypal landing page. Obviously this is all designed to push people to paypal. Given the 700 or so payment gateways, is it time to change? With regards to Omnipay, it does not look like they have done ANY updates in many years. One port to V2 was built by a customer but is not supported anywhere.

" This is inherently a fussy collation/DBA question, so unfortuantely it’s not newbie friendly. Long story short, OJS faithfully connects with whatever character set you specify in config.inc.php , but over the years MySQL has changed its default character set, and it’s possible to accidentally garble things with MySQL dump/reload cycles if you don’t specify the character set on the command line, which has sometimes happened to OJS users. The OJS serialize /unserialize functions are picky about string lengths, and a UTF-8 string that’s been incorrectly encoded in the database may trigger that. We don’t use serialize /unserialize for much and it’s generally not critical. That may be what’s happened here."

Disclaimer: I only program in assembly language! Many suggest that data should not be serialized ever if being written to a database. Seems that serialize jams content together so you can keep it in one field versus many? The follow up questions are: a) What tables have serialized data? b) Given a), we could go through and hand-edit data to match right? c) How do I write the data being unserialized to the log file? I can’t quite get the syntax correct. In this code snippet below, how do I get trigger_error to output the value of “request”? I can then adapt it to log unserialized data.

$request = Registry::get(‘request’);
if (Config::getVar(‘debug’, ‘deprecation_warnings’)) trigger_error(‘Deprecated call without request object.’);

“If you use the code formatting tools available in this forum software it’ll be easier to read; I’m not sure if you’re highlighting a specific line of code. But no, I don’t think there’s a race condition. PayPal should fulfill the payment in OJS before redirecting the user back to the article webpage.”

I apologize as I do not know how to use the code quoting tool and the forum post took decided to split my cut and paste into two pieces. Basically the comments in the code in pages/article/ArticleHandler.inc.php suggests that access is granted immediately “if the article has been paid for then forget about everything else * and just let them access the article” Could you comment on that? Is there an implied open?

Thank you!

Here is one other explanation for this…

We have talked to a few who have abandoned their purchase. Seems that many are institutional subscribers and if the ip range is (invalid, out of date, outside the exprozy range), it seems that human nature drives them to click the purchase link to see if they can get access. This results in an abandoned order on paypal and in the queue_payments folder. This could explain a large number of access attempts. TO SOLVE this, is there a way to change the wording on a screen along the way? Something like, “If you have site based access…” maybe right before the click on the pay button.

This seems reasonable as it explains why we have older users generating payments. That said, it would be great to be able to dump payment details to the log file.

As for the queue payments table, we have several cases where a PAID order is not being removed from that database. Can you point me to that code?

Thank you!!

Another piece to the puzzle.
In multiple posts it is claimed that utf-8 should be utf8 for BOTH client_charset and connection_charset. The default config.inc.php shows an inconsistency.
Can someone that BOTH entries should be utf8 with no “-”?

From the config.inc.php file we have
; Client output/input character set
client_charset = utf-8
; Database connection character set
connection_charset = utf8

Hi @radjr,

We have had several people comment that they will not use paypal as it looks “retail” in nature versus professional. Additionally, paypal “pushes” people to sign in and many don’t like that, and finally, the option to use a credit card is down on the paypal landing page. Obviously this is all designed to push people to paypal. Given the 700 or so payment gateways, is it time to change?

We intentionally wrote up PayPal support as a plugin in order to facilitate an ecosystem of payment tools rather than encouraging any one to have a monopoly. I don’t think we need to e.g. drop PayPal support, but I would like to see other contenders getting community support.

In multiple posts it is claimed that utf-8 should be utf8 for BOTH client_charset and connection_charset. The default config.inc.php shows an inconsistency. Can someone that BOTH entries should be utf8 with no “-”?

The discrepancy between utf8 and utf-8 is intentional and should be kept that way. See e.g. this topic.

As for the queue payments table, we have several cases where a PAID order is not being removed from that database. Can you point me to that code?

See this post above; it’s in the 2nd link.

Many suggest that data should not be serialized ever if being written to a database. Seems that serialize jams content together so you can keep it in one field versus many? The follow up questions are: a) What tables have serialized data? b) Given a), we could go through and hand-edit data to match right? c) How do I write the data being unserialized to the log file? I can’t quite get the syntax correct. In this code snippet below, how do I get trigger_error to output the value of “request”? I can then adapt it to log unserialized data.

We don’t use serialization much, but there are some fields where it makes sense. Rather than trying to list them out, I’d suggest starting from the warnings – you’ll need to know what the bad data is in order to fix it anyway. I think you’ll need someone with basic PHP and MySQL skills here; I can’t walk you through it over the forum.

Basically the comments in the code in pages/article/ArticleHandler.inc.php suggests that access is granted immediately “if the article has been paid for then forget about everything else * and just let them access the article” Could you comment on that? Is there an implied open?

Is there a specific problem you’re getting it? At a glance, the code makes sense to me.

Regards,
Alec Smecher
Public Knowledge Project Team

This topic was automatically closed after 13 days. New replies are no longer allowed.