The Vapnik-Chervonenkis Dimensions of Different Neural Network Architectures

    Publikation: AbschlussarbeitMasterarbeit

    15 Downloads (Pure)

    Abstract

    The Vapnik-Chervonenkis dimension, VC dimension in short, is a measure of expressivity or
    richness of a set of functions. In this thesis, we explore this concept in relation to different neural
    network architectures that use sigmoid activation functions. More specifically, we will take a
    look at classical multilayered feed-forward neural networks and at two NeuralODE architectures,
    namely Liquid Time Constant (LTC) networks and Continuous-Time Recurrent Neural Networks
    (CT-RNNs). In the latter two, the output of the network is computed by numerically solving an
    ordinary differential equation.
    For these networks, we derived upper bounds on the VC dimension, depending on the number
    of neurons, and in case of the recurrent models (LTC and CT-RNN), discretization steps. This
    was done through a method involving the number of components of the zero-set of functions
    that are dependent on the network parameters. Here various techniques relating to topology and
    geometrical analysis were used. We find a very strong dependence of the VC dimension bound
    on the number of neurons and a sizeable dependence on the number of discretization steps. The
    recurrent models had a higher bound than the classical network for the same number of neurons,
    which is partly due to the recurrent models having more parameters than the classical network.
    OriginalspracheEnglisch
    QualifikationDiplomingenieur
    Gradverleihende Hochschule
    • TU Wien
    Betreuer/-in / Berater/-in
    • Heitzinger, Clemens, Betreuer:in, Externe Person
    • Grosu, Radu, Betreuer:in, Externe Person
    Datum der Bewilligung28 Juni 2023
    PublikationsstatusVeröffentlicht - Juni 2023

    Research Field

    • Außerhalb der AIT Research Fields
    • Hybrid Power Plants

    Fingerprint

    Untersuchen Sie die Forschungsthemen von „The Vapnik-Chervonenkis Dimensions of Different Neural Network Architectures“. Zusammen bilden sie einen einzigartigen Fingerprint.

    Diese Publikation zitieren