I was trying to load the eurlex_train.txt and eurlex_test.txt.
As far as I understood they are in the LibSVM format for multilabel classification.
Using the sklearn.datasets.load_svmlight_file fails though.
I've observed that in the eurlex_train.txt file, there are 28 rows holding no label, where the newline starts with a space.
If you run the following command
cat eurlex_train.txt | grep -n "^ " | cut -d ':' -f 1
it results in 28 rows with the following line numbers in the eurlex_train.txt where the labels are missing:
95
254
511
1529
1941
1955
4031
4428
4645
4729
5233
5764
6297
6335
6705
7085
9479
9677
10001
10490
10738
10912
11676
12282
12601
13149
14169
14724
Despite this, the training using the Rust CLI (and the python wrapper too) works straight.
I've observed that a check for the presence of labels in the line are present in the omikuji/src/data.rs by the parse_xc_repo_data_line function.
Since it seems I cannot rely on the very good sklearn.datasets.load_svmlight_file, what label should I assign to those rows?
In a first simple implementation I decided to skip missing-label rows.
I was trying to load the
eurlex_train.txtandeurlex_test.txt.As far as I understood they are in the LibSVM format for multilabel classification.
Using the
sklearn.datasets.load_svmlight_filefails though.I've observed that in the
eurlex_train.txtfile, there are 28 rows holding no label, where the newline starts with a space.If you run the following command
it results in 28 rows with the following line numbers in the
eurlex_train.txtwhere the labels are missing:Despite this, the training using the Rust CLI (and the python wrapper too) works straight.
I've observed that a check for the presence of labels in the line are present in the
omikuji/src/data.rsby theparse_xc_repo_data_linefunction.Since it seems I cannot rely on the very good
sklearn.datasets.load_svmlight_file, what label should I assign to those rows?In a first simple implementation I decided to skip missing-label rows.