Differentiable fuzzy-logic layers for PyTorch — building blocks for ANFIS, Mamdani FIS, Takagi–Sugeno FIS and hybrid neuro‑fuzzy architectures (classifiers, autoencoders, anomaly detectors).
A fuzzy inference system (FIS) generally consists of three stages:
- Fuzzification — input vectors are mapped to rule-activation (firing) strengths through membership functions
μ(x). - Inference — firings are combined according to a rule base.
- Defuzzification — firings are collapsed back into a crisp output (scalar, vector, class logits, …).
torchfuzzy provides each of these steps as a standard torch.nn.Module, so any FIS — ANFIS, Mamdani, Takagi–Sugeno, TSK, custom — can be assembled as an ordinary nn.Sequential and trained end-to-end with autograd.
All fuzzification layers share a common base (FuzzyLayerBase) that parametrises each term by a Mahalanobis-like distance with a full, learnable symmetric positive-definite metric, factorised via Cholesky to remain SPD throughout training:
Different subclasses of FuzzyLayerBase then turn
pip install torchfuzzyRequirements: torch>=1.8, numpy.
Parent class. Holds learnable centers scales) and the strictly-lower triangle of rot). Supplies:
input_mask— per-term feature gate,active_mask— per-term on/off switch,prune_inactive()— physically deletes disabled terms,freeze(centers, scales, rot)— selectively freeze parameters.
Subclasses only need to implement _membership(rx2).
The classic radial-basis / Gaussian membership. Best default choice.
Heavier tails than Gaussian — useful when you want non-vanishing firing far from the cluster center (e.g. to retain gradient signal during early training).
The classical ANFIS membership function with an extra learnable shape exponent
"raw"—b = b_raw(unconstrained, may go non‑positive);"softplus"—b = softplus(b_raw) + ε(strictly positive, default);"exp"—b = exp(b_raw)(strictly positive, multiplicative).
with learnable consequents with_norm=False removes the denominator and the layer reduces to a bias-free linear map (useful for Takagi–Sugeno-style outputs).
Non-smooth, winner-takes-all style. Good fit for classification heads with one rule per class.
import torch
from torchfuzzy import FuzzyLayer
x = torch.rand(10, 2)
layer = FuzzyLayer.from_dimensions(size_in=2, size_out=4)
firings = layer(x) # (10, 4)Mamdani rule base:
import torch.nn as nn
from torchfuzzy import FuzzyLayer, DefuzzyNWLayer
mamdani_fis = nn.Sequential(
FuzzyLayer.from_dimensions(input_dim, n_rules, trainable=True),
DefuzzyNWLayer.from_dimensions(n_rules, output_dim, with_norm=True),
)import torch
import torch.nn as nn
from torchfuzzy import FuzzyBellLayer, DefuzzyNWLayer
class ANFIS(nn.Module):
def __init__(self, n_in, n_rules, n_out):
super().__init__()
self.fuzzify = FuzzyBellLayer.from_dimensions(
n_in, n_rules, b_parametrization="softplus"
)
self.defuzz = DefuzzyNWLayer.from_dimensions(
n_rules, n_out, with_norm=True # Nadaraya–Watson normalization
)
def forward(self, x):
firings = self.fuzzify(x) # (B, n_rules)
return self.defuzz(firings) # (B, n_out)
model = ANFIS(n_in=4, n_rules=16, n_out=1)
y = model(torch.randn(32, 4))For a first-order TS system replace the constant consequent by a per-rule linear function of the inputs:
import torch
import torch.nn as nn
from torchfuzzy import FuzzyLayer
class TSK1(nn.Module):
def __init__(self, n_in, n_rules, n_out):
super().__init__()
self.fuzzify = FuzzyLayer.from_dimensions(n_in, n_rules)
self.consequent = nn.Linear(n_in, n_rules * n_out)
self.n_rules, self.n_out = n_rules, n_out
def forward(self, x):
w = self.fuzzify(x) # (B, R)
w = w / w.sum(dim=-1, keepdim=True).clamp_min(1e-9)
y = self.consequent(x).view(-1, self.n_rules, self.n_out)
return (w.unsqueeze(-1) * y).sum(dim=1) # (B, n_out)from torchfuzzy import FuzzyCauchyLayer, DefuzzyMaxLayer
classifier = nn.Sequential(
FuzzyCauchyLayer.from_dimensions(n_features, n_rules),
DefuzzyMaxLayer.from_dimensions(n_rules, n_classes),
)encoder = torchvision.models.resnet18(num_classes=32)
head = nn.Sequential(
FuzzyLayer.from_dimensions(32, n_rules),
DefuzzyNWLayer.from_dimensions(n_rules, n_classes),
)
model = nn.Sequential(encoder, head)active_mask is a buffer of shape (size_out,). Zeroing out an entry forces the corresponding firing to zero without removing the parameters:
layer = FuzzyLayer.from_dimensions(size_in=4, size_out=8)
x = torch.randn(16, 4)
layer.set_active([1, 3, 5], active=False)
assert layer.n_active == 5
out = layer(x)
assert (out[:, [1, 3, 5]] == 0).all()
layer.reset_mask() # re-enable allAfter soft-disabling rules you can physically shrink the layer. All parameters (centers, scales, rot, and, for FuzzyBellLayer, _b_raw) are resliced in-place:
with torch.no_grad():
mean_act = layer(x_val).mean(dim=0) # (size_out,)
dead = (mean_act < 1e-3).nonzero().view(-1).tolist()
if dead:
layer.set_active(dead, active=False)
layer.prune_inactive()
print(layer.size_out, layer.centers.shape)Typical uses:
- removing rules that never fire,
- post-training compression,
- knowledge-distillation / structured pruning schedules.
Each term can be restricted to a subset of input features via input_mask of shape (size_out, size_in):
# Rule 0 sees features {0,1}; rule 1 — {2,3}; rule 2 — {0,2}.
mask = torch.tensor([
[1, 1, 0, 0],
[0, 0, 1, 1],
[1, 0, 1, 0],
], dtype=torch.float32)
layer = FuzzyLayer.from_centers(
torch.randn(3, 4),
input_mask=mask,
)This is equivalent to forcing the corresponding rows of each
layer = FuzzyBellLayer.from_dimensions(8, 16)
# Freeze centers and bell exponents, keep scales/rot trainable
layer.freeze(centers=True, powers=True)
# Fine-grained control
layer.set_requires_grad_centroids(False)
layer.set_requires_grad_scales(True)
layer.set_requires_grad_rot(True)A = layer.get_transformation_matrix() # (size_out, n, n) Cholesky factor
M = layer.get_metric_matrix() # (size_out, n, n) SPD metric
eig = layer.get_transformation_matrix_eigenvals() # all > 0
centers = layer.get_centroids()- Variational Autoencoders with Fuzzy Inference (Russian)
- Conditional Variational Autoencoders with Fuzzy Inference
If you use torchfuzzy in academic work, please cite:
@InProceedings{10.1007/978-3-031-77411-9_9,
author = "Gurov, Yury and Khilkov, Danil",
editor = "Kovalev, Sergey and Kotenko, Igor and Sukhanov, Andrey and Li, Yin and Li, Yao",
title = "Conditional Variational Autoencoders with Fuzzy Inference",
booktitle = "Proceedings of the Eighth International Scientific Conference 'Intelligent Information Technologies for Industry' (IITI'24), Volume 2",
year = "2024",
publisher = "Springer Nature Switzerland",
address = "Cham",
pages = "91--103",
isbn = "978-3-031-77411-9"
}Related work worth citing when using specific components:
@article{jang1993anfis,
title = {ANFIS: adaptive-network-based fuzzy inference system},
author = {Jang, Jyh-Shing Roger},
journal = {IEEE Transactions on Systems, Man, and Cybernetics},
volume = {23}, number = {3}, pages = {665--685}, year = {1993}
}
@article{takagi1985fuzzy,
title = {Fuzzy identification of systems and its applications to modeling and control},
author = {Takagi, Tomohiro and Sugeno, Michio},
journal = {IEEE Transactions on Systems, Man, and Cybernetics},
volume = {SMC-15}, number = {1}, pages = {116--132}, year = {1985}
}MIT (see LICENSE).