Para openSUSE Tumbleweed execute o seguinte como root:
zypper addrepo https://download.opensuse.org/repositories/home:mslacken:ml/openSUSE_Tumbleweed/home:mslacken:ml.repo
zypper refresh
zypper install python-tokenizers
Para openSUSE Factory ARM execute o seguinte como root:
zypper addrepo https://download.opensuse.org/repositories/home:mslacken:ml/openSUSE_Factory_ARM/home:mslacken:ml.repo
zypper refresh
zypper install python-tokenizers
Para 16.0 execute o seguinte como root:
zypper addrepo https://download.opensuse.org/repositories/home:mslacken:ml/16.0/home:mslacken:ml.repo
zypper refresh
zypper install python-tokenizers
Para 15.6 execute o seguinte como root:
zypper addrepo https://download.opensuse.org/repositories/home:mslacken:ml/15.6/home:mslacken:ml.repo
zypper refresh
zypper install python-tokenizers