Data Toolkit - Advanced Customizations

[yannispoulakis]

Για καθε Classification, Regression, Clustering job θα πρεπει να μπορει ο χρηστης να τα παραμετροποιει με καποια συγκεκριμενα attributes.

Tasks:

• Να ενημερωθει η διεπαφη του Process Modelling ωστε οταν ο χρηστης επιλεγει ενα απο τα τρια παραπανω nodes να εμφανιζονται τα αναλογα πεδια προς συμπληρωση
• Μονο ο χρήστης με δικαιωματα Data Scientist θα μπορει να εχει προσβαση στο toolkit
• Να μπορει η εφαρμογη να κραταει state ωστε να μενουν οι επιλογες που πληκτρολογηθηκαν σε καθε node
• Να γίνεται type checking σε ολα τα πεδια
• Να ενημερωθει το τελικο JSON playbook οπως περιγραφεται παρακατω

[adreaskar]

Logistic Regression

• maxIter: int = 100
• regParam: float = 0.0
• elasticNetParam: float = 0.0
• tol: string = 1e-06
• fitIntercept: bool = True
• threshold: float = 0.5
• standardization: bool = True
• aggregationDepth: int = 2
• family: str = auto, binomial, multinomial
• maxBlockSizeInMB: float = 0.0

Decision Tree Classifier

• maxDepth: int = 5
• maxBins: int = 32
• minInstancesPerNode: int = 1
• minInfoGain: float = 0.0
• maxMemoryInMB: int = 256
• cacheNodeIds: bool = False
• checkpointInterval: int = 10
• impurity: string = entropy, gini
• minWeightFractionPerNode: float = 0.0

Random Forest Classifier

• maxDepth: int = 5
• maxBins: int = 32
• minInstancesPerNode: int = 1
• minInfoGain: float = 0.0
• maxMemoryInMB: int = 256
• cacheNodeIds: bool = False
• checkpointInterval: int = 10
• impurity: string = entropy, gini
• numTrees: int = 20
• subsamplingRate: float = 1.0
• minWeightFractionPerNode: float = 0.0

GBTClassifier

• maxDepth: int = 5
• maxBins: int = 32
• minInstancesPerNode: int = 1
• minInfoGain: float = 0.0
• maxMemoryInMB: int = 256
• cacheNodeIds: bool = False
• checkpointInterval: int = 10
• maxIter: int = 20
• stepSize: float = 0.1
• subsamplingRate: float = 1.0
• validationTol: float = 0.01
• minWeightFractionPerNode: float = 0.0

MLP Classifier

• maxIter: int = 100
• tol: string = 1e-06
• blockSize: int = 128
• stepSize: float = 0.03
• solver: string = l-bfgs | gd

Linear SVC

• maxIter: int = 100
• regParam: float = 0.0
• tol: string = 1e-06
• fitIntercept: bool = True
• standardization: bool = True
• threshold: float = 0.0
• aggregationDepth: int = 2
• maxBlockSizeInMB: float = 0.0

Linear Regression

• maxIter: int = 100
• regParam: float = 0.0
• elasticNetParam: float = 0.0
• tol: string = 1e-06
• fitIntercept: bool = True
• standardization: bool = True
• solver: string = auto | normal | l-bfgs.
• aggregationDepth: int = 2
• loss: string = squaredError | huber
• epsilon: float = 1.35
• maxBlockSizeInMB: float = 0.0

DecisionTreeRegressor

• maxDepth: int = 5
• maxBins: int = 32
• minInstancesPerNode: int = 1
• minInfoGain: float = 0.0
• maxMemoryInMB: int = 256
• cacheNodeIds: bool = False
• checkpointInterval: int = 10
• minWeightFractionPerNode: float = 0.0

RandomForestRegressor

• maxDepth: int = 5
• maxBins: int = 32
• minInstancesPerNode: int = 1
• minInfoGain: float = 0.0
• maxMemoryInMB: int = 256
• cacheNodeIds: bool = False
• checkpointInterval: int = 10
• subsamplingRate: float = 1.0
• numTrees: int = 20
• minWeightFractionPerNode: float = 0.0

GBTRegressor

• maxDepth: int = 5
• maxBins: int = 32
• minInstancesPerNode: int = 1
• minInfoGain: float = 0.0
• maxMemoryInMB: int = 256
• cacheNodeIds: bool = False
• subsamplingRate: float = 1.0
• checkpointInterval: int = 10
• lossType: string = squared | absolute
• maxIter: int = 20
• stepSize: float = 0.1
• validationTol: float = 0.1

KMeans

• k: int = 2
• initMode: string = k-means|| | random (k-means|| IS NOT A TYPO)
• initSteps: int = 2
• tol: string = 0.0001
• maxIter: int = 20
• distanceMeasure: string = 'euclidean' | cosine

Latent Dirichlet allocation

• maxIter: int = 20
• checkpointInterval: int = 10
• k: int = 10
• optimizer: str = online | em
• learningOffset: float = 1024.0
• learningDecay: float = 0.51
• subsamplingRate: float = 0.05
• optimizeDocConcentration: bool = True
• keepLastCheckpoint: bool = True

Gaussian Mixture

• k: int = 2
• tol: string = 0.01
• maxIter: int = 100
• aggregationDepth: int = 2

[adreaskar]

Το τελικο playbook θα εχει την εξης μορφη

{ 
    algorithm: <string>,
    column: <string>,
    params : {
        maxDepth: <>,
        maxBins: <>,
        ....
        Αν automl: true τότε υπάρχουν επίσης:
        max-trials: <int>
        meta-learning: <boolean>
    },
    automl: <boolean>,
}

[adreaskar]

Τελικο playbook update (αφορα την ονομασια των αλγοριθμων):

CLASSIFICATION

• logistic regression - logisticregression
• decision tree classifier - decisiontree
• random forest classifier - randomforest
• gradient boosted tree classifier - gbt
• multilayer perceptron - mlp
• linear support vector machine - linearsvc

REGRESSION

• linear regression - linearregression
• decision tree regression - decisiontree
• random forest regression - randomforest
• gradient boosted tree regression - gbt

CLUSTERING

• kmeans - kmeans
• latent dirichlet allocation - lda
• gaussian mixture model - gmm

[adreaskar]

Ολα τα tasks εχουν ολοκληρωθει για αυτο το issue, το προχωραω στο integration phase

[karamolegkos]

Αυτό το Component θα γίνει Done όταν εξεταστεί σε συνδιασμό με το AutoML.
Αυτήν την στιγμή περνάει από ένα big fixing phase το οποίο έχει συζητηθεί από εμένα και τον @yannispoulakis .