Abstract
The VapnikChervonenkis dimension, VC dimension in short, is a measure of expressivity or
richness of a set of functions. In this thesis, we explore this concept in relation to different neural
network architectures that use sigmoid activation functions. More specifically, we will take a
look at classical multilayered feedforward neural networks and at two NeuralODE architectures,
namely Liquid Time Constant (LTC) networks and ContinuousTime Recurrent Neural Networks
(CTRNNs). In the latter two, the output of the network is computed by numerically solving an
ordinary differential equation.
For these networks, we derived upper bounds on the VC dimension, depending on the number
of neurons, and in case of the recurrent models (LTC and CTRNN), discretization steps. This
was done through a method involving the number of components of the zeroset of functions
that are dependent on the network parameters. Here various techniques relating to topology and
geometrical analysis were used. We find a very strong dependence of the VC dimension bound
on the number of neurons and a sizeable dependence on the number of discretization steps. The
recurrent models had a higher bound than the classical network for the same number of neurons,
which is partly due to the recurrent models having more parameters than the classical network.
richness of a set of functions. In this thesis, we explore this concept in relation to different neural
network architectures that use sigmoid activation functions. More specifically, we will take a
look at classical multilayered feedforward neural networks and at two NeuralODE architectures,
namely Liquid Time Constant (LTC) networks and ContinuousTime Recurrent Neural Networks
(CTRNNs). In the latter two, the output of the network is computed by numerically solving an
ordinary differential equation.
For these networks, we derived upper bounds on the VC dimension, depending on the number
of neurons, and in case of the recurrent models (LTC and CTRNN), discretization steps. This
was done through a method involving the number of components of the zeroset of functions
that are dependent on the network parameters. Here various techniques relating to topology and
geometrical analysis were used. We find a very strong dependence of the VC dimension bound
on the number of neurons and a sizeable dependence on the number of discretization steps. The
recurrent models had a higher bound than the classical network for the same number of neurons,
which is partly due to the recurrent models having more parameters than the classical network.
Originalsprache  Englisch 

Qualifikation  Diplomingenieur 
Gradverleihende Hochschule 

Betreuer/in / Berater/in 

Datum der Bewilligung  28 Juni 2023 
Publikationsstatus  Veröffentlicht  Juni 2023 
Research Field
 Außerhalb der AIT Research Fields
 Hybrid Power Plants