Character Encoding Issue ftfy

[sarfaraz]

• Application Version OJS 3.1.2.4
• Description of issue : Incorrect Characters
• Steps you took leading up to the issue :
• What you tried to resolve the issue : Research went through hours of stackflow and forum hints and tips

Good Morning

Issue:
I have a xx.sql that has some encoding issues.
eg:
represent” in Xhosa)Â

I was advised to use ftfy to fix (ftfy · PyPI)
I have installed (pip install ftfy) but for the sake of me cannot comprehend how to use it for example:

in terminal these are the commands i run :

python
import ftfy
ftfy --output=xx.clean.sql xx.orig.sql (not sure how to point to the location of xx.orig.sql)

---

I feel like it is something simple that i am missing any help with this would be appreciated.

Thank you

[sarfaraz]

These are directions for Ubuntu 18.04 that i used for my solution

apt install python3
apt install python-pip3
pip3 install ftfy

cd /usr/local/lib/python3.6/dist-packages/ftfy
vim +100 cli.py

#Changes to file bold part

if args.preserve_entities:
unescape_html = False
else:
unescape_html = ‘auto’

config = TextFixerConfig(
    unescape_html=unescape_html,
    normalization=normalization**,**
    **uncurl_quotes=False**
)

try:
    for line in fix_file(
        file,
        encoding=encoding,
        config=config

---

/usr/local/lib/python3.6/dist-packages# ftfy --output=client.clean.sql xxx.sql
#copied the file to same directory