Combining Optimization Objectives: New Modeling Attacks on Strong PUFs

. Strong Physical Unclonable Functions (PUFs), as a promising security primitive, are supposed to be a lightweight alternative to classical cryptography for purposes such as device authentication. Most of the proposed candidates, however, have been plagued by modeling attacks breaking their security claims. The Interpose PUF (iPUF), which has been introduced at CHES 2019, was explicitly designed with state-of-the-art modeling attacks in mind and is supposed to be impossible to break by classical and reliability attacks. In this paper, we analyze its vulnerability to reliability attacks. Despite the increased diﬃculty, these attacks are still feasible, against the original authors’ claim. We explain how adding constraints to the modeling objective streamlines reliability attacks and allows us to model all individual components of an iPUF successfully. In order to build a practical attack, we give several novel contributions. First, we demonstrate that reliability attacks can be performed not only with covariance matrix adaptation evolution strategy (CMA-ES) but also with gradient-based optimization. Second, we show that the switch to gradient-based reliability attacks makes it possible to combine reliability attacks, weight constraints, and Logistic Regression (LR) into a single optimization objective. This framework makes modeling attacks more eﬃcient, as it exploits knowledge of responses and reliability information at the same time. Third, we show that a diﬀerentiable model of the iPUF exists and how it can be utilized in a combined reliability attack. We conﬁrm that iPUFs are harder to break than regular XOR Arbiter PUFs. However, we are still able to break (1,10)-iPUF instances, which were originally assumed to be secure, with less than 10 7 PUF response queries.


Introduction
Process variations at transistor-level are a primarily detrimental factor in integrated circuit (IC) manufacturing as they negatively impact performance and yield-rates. They can, however, be used as a source of entropy for Physical Unclonable Functions (PUFs), which provide chip "fingerprints" derived from their inherent physical features [HYKD14, RH14, LLG + 04]. These fingerprints can be used in different protocols and applications to achieve security goals such as device authentication.
PUFs act as physical functions that behave uniquely and ideally generate unpredictable outputs or responses when queried for inputs or challenges. They are mainly divided into two categories based on the size of their challenge space. Weak PUFs have a very limited challenge space whose size is only polynomial in the PUF area. This type of PUF can be used to derive cryptographic keys which are supposedly better secured by being encoded in the physical structure of the PUF instead of being stored in traditional non-volatile memory. In contrast, so-called Strong PUFs possess an exponentially large challenge space, which enables the use of challenge-response protocols without costly traditional cryptographic primitives. They are supposed to be a lightweight solution, for example, for radio-frequency identification (RFID)-tag authentication. The attack vectors for both types are quite different. Weak PUF responses are used as a source of key material to be used in classical cryptographic protocols and are not directly accessible to an attacker [MVHV12,YSS + 12]. Therefore, an attacker can either target the employed protocol and helper data algorithms [DV14,DGSV14,BWG15,Bec17] or use side-channel attacks to extract key material [KS10, MSSS11, HBNS13, MHH + 13]. Strong PUF responses, on the other hand, are directly exposed to an attacker. In the standard adversarial model, it is either assumed that an attacker can passively eavesdrop a number of protocol runs or is even able to query responses directly from the PUF for arbitrary challenges. The main attacker goal is to build clones of Strong PUF instances which are able to generate correct responses for arbitrary challenges. Mathematical cloning is the most common way to meet this goal and so far, powerful modeling attacks have prevented Strong PUFs from being adopted in real applications [RSS + 10, RSS + 13a, Bec15, Del19, WBM + 19]. However, Strong PUFs are also susceptible to side-channel attacks [DV13, RXS + 14, BK + 14, TDF + 14, AM20].
Thus far, Strong PUF candidates predominantly rely on delay-based Arbiter PUFs (APUFs) as their main building blocks for PUF constructs and protocols [GCVDD02, SD07, MKP08, VHKM + 12, DPGV15, YHD + 16, SMCN17,Del19]. These can be modeled by a linear function which is the basis for a host of attacks that use collected challenge-response pairs (CRPs) or challenge-reliability pairs to build complete mathematical clones of the complete PUF construction [RSS + 13a, Bec15]. Utilizing information about the reliability of responses has greatly amplified the threat posed by modeling attacks, by reducing the scaling of the required training set size from exponential to linear in the number of APUFs [Bec15]. To counter this reliability-based attack, protocol-level countermeasures have been proposed that prevent an attacker from collecting reliability information [YHD + 16]. This is achieved by randomizing the challenges on the PUF device. A software model of the PUF is necessary to test the authenticity of PUF responses. Nguyen et al. [NSJ + 19] proposed a new lightweight APUF-based construction which uses a domino structure to be especially resistant against reliability-based modeling attacks without protocol level countermeasures. In brief, the Interpose PUF (iPUF) consists of two XOR Arbiter PUFs, the x-(upper) and y-(lower) PUF. The x-XOR PUF response is used as an additional challenge bit for the y-XOR PUF. In their security analysis, the authors claim that the constituent APUFs of the x-XOR PUF cannot be recovered by reliability attacks. Therefore, given that the y-XOR PUF is large enough, the complete iPUF is supposed to be secure. However, recently direct modeling attacks on the iPUF have been proposed, raising first doubts regarding its security [SBC19,WMP + 20] In this work, we show that the individual x-APUFs still show enough correlation with the PUF reliability and can indeed be learned given enough data. In order to provide a practical attack, we develop a reliability attack in the gradient-based optimization framework which allows us to conveniently combine different optimization goals into a single objective. This new attack is more efficient than the commonly used CMA-ES-based approach. While using multiple optimization objectives and constraints in machine learning is nothing new [HTF09], it is the first time that this technique has been applied to machine learning attacks on APUF-based constructions. We strongly believe that such a strategy has application outside of reliability-based attacks used in this paper and will inspire other modeling attacks with different objectives in the future.

Related Work
The field of modeling attacks on APUF-based Strong PUFs has been heavily influenced by two approaches. Logistic regression (LR) as introduced by Rührmair et al. [RSS + 10, RSS + 13a] has been shown to work well for attacking XOR PUFs, leaving only very large instances secure that can hardly be efficiently realized in hardware. Becker [Bec15] showed that attacks can be performed with covariance matrix adaptation evolution strategy (CMA-ES) using much smaller datasets if reliability information is available. Our work uses ideas from both approaches. More information about them are given in Section 2.3. Nguyen et al. [NSJ + 19] analyzed the iPUF in light of both approaches and found the construction to be secure. Afterward, new publications showed that the iPUF is not quite as hard to break as initially assumed. Santikellur et al. [SBC19] showed deep learning results up to (4,4)-iPUFs with a high accuracy of more than 0.97 and a moderate CRP set size of 319, 000. Wisiol et al. [WMP + 20] showed that the original LR attack can be adapted to the iPUF by "splitting" it, i.e., considering its two XOR PUFs individually. The x-XOR PUF and y-XOR PUF can be learned iteratively, which effectively reduces the security of a (k,k)-iPUF to a k-XOR PUF. While we show that a differentiable model of the iPUF exists and can be used for a regular LR attack, we found that this approach does not work well for larger iPUF instances because the optimization converges to local optima, in which the y-PUF has been accurately learned and the x-PUF remains essentially random (see Section 6.5 for more details). Therefore, we believe that the splitting technique is the de facto state-of-the-art-approach in terms of efficiency, if no reliability information is available. If, however, reliability information can be used, our novel contributions lead to a more efficient attack.
In addition to the mentioned attacks, there is a line of research that draws from learning theory and examines model building the Probably Approximately Correct (PAC) framework [GTS16,GTFS16,GTS15]. In particular, in [GTFS16], Ganji et al. introduced a weak PAC learning algorithm, proved its equivalence to a strong PAC algorithm, and applied it to Bistable Ring PUFs. In [GTS18], Ganji et al. enhanced their PAC learning methodology. They introduced an improper PAC learning algorithm, making use of Fourier analysis of PUFs in Boolean function representation. This approach can tolerate noise and is independent of the specific model delivered by the algorithm, making it the strongest PAC learning algorithm applied in the context of PUFs.
Several approaches for increasing the security of Strong PUFs against modeling attacks have been explored. Sahoo et al. [SMCN18] proposed a multiplexer-based alternative to the XOR Arbiter PUF. It was claimed that this construction provides more reliability and security but as argued in [NSJ + 19] still falls victim to reliability attacks. Wisiol et al. [WGM + 17] suggested to increase the reliability of individual APUFs by averaging their response, which in turn makes larger XOR Arbiter PUFs feasible. Drawbacks of this approach are the hardware overhead and its vulnerability to reliability attacks.
Noise-bifurcation as introduced by Yu et al. [YMVD14] is a technique that introduces additional noise into the training set of the attacker which, however, can be mitigated by increasing the training set size [TB15]. The lockdown protocol by Yu et al. [YHD + 16] addresses reliability attacks by preventing the adversary from collecting reliability information. This is achieved by introducing mutual-authentication due to which the same challenge cannot be queried repeatedly, unless the challenger is already in possession of a valid PUF model.

Contributions
In this paper, we present several novel contributions in the scope of modeling attacks: • We show that the iPUF is, against claims in its initial security analysis, still susceptible to reliability modeling attacks. In particular, we can break the (1,10)-iPUF, which was proposed as a secure instantiation, with a training set size that is orders of magnitude lower than what would be required for LR-based modeling attacks.
• In order to facilitate this attack, we improve the reliability attack on XOR Arbiter PUFs by Becker [Bec15] by combining multiple objectives and constraints. This approach significantly improves the attack performance and can in general be applied to other APUF-based constructions. In particular, we show how one can add constraints to learning objectives in a reliability-based attack such that all APUF instances can be found in a single run. This is especially important for constructs with many individual APUF instances, as it prevents the algorithm from always converging to the same APUF instance. We further improve upon the attack by combining the objective used in direct modeling attacks with those of the reliability based attack.
• In previous work, reliability attacks have always been performed using the CMA-ES optimization algorithm. We show that utilizing gradient-based optimization is a good alternative. We use a modern machine learning library (PyTorch) for our implementation. •

Background
In this chapter, we review the mathematical model of Arbiter-based PUFs and give an overview of the state-of-the-art modeling attacks, LR and CMA-ES reliability attacks.

Notation
In this paper, we denote scalar values with italic characters (e.g., x). Vectors are printed in lower-case bold (e.g., v) and matrices are set in upper-case bold (e.g., M ). To access individual elements and slices of vectors and matrices, we use the square bracket notation. Slices are selected by the first and last index, divided by a colon. If either start or end index is omitted, then the slice starts at the first element or ends at the last element, respectively. The index is one-based, i.e., the first element is indexed by [1]. If the start and end index are not explicitly given in a sum, it is assumed that the sum runs over the maximum range of values (which can for example be the number of elements in a vector). Variable names, that are consistently used in the next sections, are listed in Table 1. Subscripts are used to give additional context for variables.

PUF Models
In this section, we briefly describe the APUF and two constructions, the XOR APUF and the iPUF which make use of multiple individual APUFs to increase the resistance against modeling attacks. Our description includes the delay-based mathematical model of the APUF that has been introduced by Lim [Lim04] for APUF analysis.
Arbiter PUFs consist of n consecutive stages that carry two paths, see Figure 1. Each stage has an associated challenge bit, which determines whether or not the stage flips the paths. To query the PUF, a rising signal is applied two both paths. The upper and lower signal race through all stages. At the end, an arbiter measures which of the two signals arrives first and translates the result in the binary PUF response r.
Lim [Lim04] proposed a linear additive model that captures the APUF behavior properly. This model requires the map f(c) = φ of the applied challenge c of length n to a feature or parity vector φ of length n + 1: A vector of delay values, which are also later referred to as weights, w of length n + 1 encodes all information that is required to compute responses of an APUF [Lim04]: Note that the main goal of PUF modeling attacks is learning these weights w. Additionally, a Gaussian-distributed term d noise ∼ N (0, σ 2 ) can be added to this model to consider transient environmental noise which inevitably affects actual PUF implementations: XOR Arbiter PUFs are the most common APUF-based construction in which a non-linear XOR sum is computed over individual APUF responses to increase modeling difficulty. Let the weights of all k individual APUFs be the row vectors of matrix W .
Then, XOR APUFs can be modeled as a multiplication of individual APUF delays, with the sign of the product being the response: Interpose PUFs, as shown in Figure 1, consist of two distinct XOR APUFs, which are labeled with x and y for the upper and lower layer. The upper layer response r x is added as an additional challenge bit into the challenge vector c at index i which is then applied to the lower layer. The final PUF response bit is the response r y of the lower layer XOR APUF. The iPUF is parametrized by two matrices W x and W y , one for each layer, and can be mathematically described as: The insertion index i is a free design parameter, however, according to the security analysis of Nguyen et al. [NSJ + 19], the best position is in the middle of the challenge vector and we use this position as the default in this work. Hence, we also later refer to halves of weight vectors which are the two sets of elements that are divided by index i.

Modeling Attack Background
In the regular attacker model, that is typically used [RSS + 10, TB15, NSJ + 19, WMP + 20], the goal is to find a function that predicts responses correctly for arbitrary challenges. It is assumed that the attacker has access to a device for a limited time and can query the PUF for chosen challenges 1 . Based on the collected set of CRPs, she then tries to find a model whose responses pass as authentic in future protocol runs. The required size of this set is a measure of the security of a Strong PUF construction. The two most important categories of modeling attacks on APUF-based constructions are direct-modeling and reliabilitybased attacks. Direct-modeling, which is also referred to as classical modeling attacks by Nguyen et al. [NSJ + 19], only uses challenge-response pairs to learn the PUF behavior and is, therefore, more generic than reliability-attacks, which observe multiple responses 2 for each challenge and use this additional information to more efficiently-built accurate models. For both categories, there is a standard choice for the modeling employed algorithm, LR and CMA-ES, respectively. In the following, we briefly review both approaches which are the basis for our new attack.

Direct-Modeling Attack Using Logistic Regression
It has been shown by Lim [Lim04] that APUFs can be described by a linear model which makes them trivially learnable. XOR APUFs were introduced by Suh and Devadas [SD07] as a harder-to-learn non-linear composition of single APUFs, but Rührmair et al. [RSS + 10] showed that these can be attacked with LR. This approach uses gradient descent to find model parameters and, therefore, requires a differentiable model W that maps a challenge c to a predicted response 3 based on parameters W . The binary cross-entropy function loss bin is used to measure how well a predicted response r p matches the actual binary response r a : loss bin (r p , r a ) = −r a · log(r p ) − (1 − r a ) · log(1 − r p ) (9) If a training set, consisting of a challenge matrix C, corresponding feature vector matrix Φ, and a response vector r, is given, one can compute the total loss of the training set for a given model parameter: The total loss is iteratively minimized by gradient descent. For this, the gradient loss total ∂W is evaluated and used to determine an update direction and magnitude for W . Gradient descent is not guaranteed to find a global minimum if the optimized function is non-linear which is the case, for example, for XOR PUFs. Instead, the algorithm only converges with a certain probability to a good optimum and usually has to be run multiple times to achieve successful learning. Different optimization algorithms for the weight update exist. RPROP [RB93], which computes the complete gradient in each step, has been used by Rührmair The LR approach has been shown to work well [RSS + 10, TB15] for reasonably sized XOR APUF instances, even though the required training set size scales exponentially with the number of APUFs k.

Reliability-based Attack Using CMA-ES
The potential of using reliability-information 4 for PUF modeling has been shown by Delvaux and Verbauwhede [DV13] and was later developed by Becker [Bec15] into a comprehensive attack on XOR APUFs. The number of required CRPs for this reliability attack only increases linearly as opposed to exponentially with the number of XORs k and is hence much more efficient. The underlying observation of the attack is that the overall reliability of the PUF output linearly correlates with the reliability of the output of single APUF. The attack as proposed by Becker [Bec15] requires two measures of reliability, one for queried PUF responses and one for the model predictions. The model consists of a single candidate APUF, described by delay vector w. For a single challenge c and associated feature vector φ, the magnitude of d = wφ is indicative for how reliable the corresponding response is. A binary model reliability measure, parametrized with an error boundary , is given as: For measuring PUF reliability, the PUF is queried for each challenge c for a number of l repetitions and the corresponding responses are collected in a vector r. Then, scalar h gives a per-challenge reliability measure: The goal of the attack is then to find accurate values for w and that show a high Pearson correlation over the whole training set between h and h. Becker [Bec15] used CMA-ES, a gradient-free black-box optimization method, to converge to a solution. CMA-ES is a based on the concept of evolution. In each iteration, a set of candidate solutions is drawn from a multivariate Gaussian distribution. The candidates are ranked according to how well they minimize the target function. A subset of the best solutions is used to update the mean and covariance matrix of the Gaussian distribution for the next iteration. These updates guide the optimization process into a useful direction. The number of candidates and the size of the subset, that determines the new mean and covariance matrix, are referred to as λ and µ. The optimization process is non-deterministic and its outcome depends on the random values drawn during optimization. The process of optimization has to be repeated several times to gain candidates for all individual APUFs of the attacked XOR APUF.

Reliability-based Attacks on the Interpose PUF
The iPUF was explicitly designed by Nguyen et al.
[NSJ + 19] to provide resistance against reliability-based attacks. They state, as a requirement for the CMA-ES reliability attack to work, that "each APUF must contribute equal reliability information to the output". Based on this statement, a mathematical proof is provided to show that x-APUFs and y-APUFs do contribute differently. The conclusion, therefore, is made that the iPUF should be secure against the CMA-ES reliability attack. However, in this section, we will show that this conclusion is incorrect. If individual APUFs contribute differently to the reliability information, they can still be learned.

Correlation between Overall Reliability and Individual APUFs
In the CMA-ES reliability attack paper [Bec15], it was noted that when attacking an XOR APUF, some APUF instances are learned more often than others. This resulted in a larger number of required modeling runs to fully learn all individual Arbiter PUFs. To better understand this observation, we simulated 1000 unique 64-stage 8-XOR APUFs and 100, 000 challenge and responses with a noise variance of σ 2 = 0.5 which equals a mean accuracy of 0.97 for each APUF. We then computed the absolute delay difference h * for each challenge and each APUF using the APUF delay model (mentioned in Section 2.2). Next, we computed the correlation coefficient between this absolute delay difference h * and the overall XOR APUF reliability. The latter was computed on the basis of 10 PUF response queries. It is known that the absolute delay difference h * of a PUF is correlated with the reliability (see e.g. [DV13]) and this fact is used in a reliability attack to learn the PUF models. The absolute delay difference can be computed with: We repeated this computation with an equivalent amount of random APUF instances whose correlation between model reliability and XOR APUF reliability was collected. Figure 2a shows the histograms of the resulting correlation coefficients. One can clearly see that the actual individual APUFs correlate to the reliability, unlike the random PUFs. However, we can also see that the distribution of the correlation has a significant variance and that . Additional, the correlation to random APUF instances is given. Both PUFs use APUFs with 64 stages and a mean APUF accuracy of 0.97. The correlations were computed for 100, 000 challenge-reliability pairs. some instances have a larger or smaller correlation than others. We also performed the same kind of experiment for a population of (4,4)-iPUFs. The simulation can be performed equivalently for the x-APUF instances, because their full challenge is known. The same, however, is not true for the y-APUF instances as the challenge bit at position i is not known as it depends on the x-APUF. In our modeling attack, we therefore omitted the PUF stage at position i. We call this delay vector w y-red reduced, as it has one stage less: A further point, that is important for an actual attack on the iPUF, is the fact that there is some ambiguity in regard to vector w y-red . The inclusion of the x-PUF response r x into challenge c y effectively leads to the bit-wise inversion of the first half of the feature vector for the y-PUF for approximately half of all challenges (when the x-PUF response is 1).
Since an attacker does not know the output of the x-APUF we have to approximate the y-APUF output by assuming that either the x-APUF is always 1 or 0. Therefore each y-PUF has actually two candidates, one is w y-red and the other is w y-red in which the first half of the delay vector is inverted: Histogram 2b shows the correlation between the iPUF reliability and the individual APUFs from the x-PUFs and the flipped and not flipped models from the y-PUF. One can clearly observe that the correlation of x-APUFs on average is smaller than that of the y-APUFs. However, it is important to note that some of the x-APUFs actually have a larger correlation than many of the y-APUFs. Hence, one cannot conclude that x-APUFs will always be harder to learn than y-PUFs based on the correlation analysis. Note that the correlation distribution between x-APUFs and y-APUFs depends on the level of noise such that the difference can increase or decrease for different noise levels. Furthermore, in the analysis performed by Nguyen et al.
[NSJ + 19] the fact that in a CMA-ES only partially correct y-APUF models w y-red and w y-red are learned was not considered. Therefore, the difference between x-APUF correlation and y-APUF correlation is smaller in practice than assumed by Nguyen [NSJ + 19]. Note that the correlation coefficient for both y-PUFs and x-PUFs is smaller than that of the XOR-PUF, indicating that XOR APUFs should be learnable with less CRPs than iPUFs with an equivalent amount of individual APUFs.

Interplay between Correlation and Convergence Rate
To verify in how far an APUF's correlation coefficient can directly be linked to the convergence probability during a CMA-ES attack, we performed the attack on a simulated 8-XOR PUF and (4,4)-iPUF with 1000 independent CMA-ES runs and counted to which APUF a run converged how often. We used a noise level of σ = 0.5 which resulted in roughly 97% reliability for an individual 64-stage Arbiter PUF. The result is depicted in Figure 3. Looking at the 8-XOR PUF in Figure 3a, one can clearly see that the correlation and convergence rate are strongly related to each other and that some instances are harder to learn than others. One can also observe the relationship between correlation and convergence rate for the (4,4)-iPUF in Figure 3b. Recall that for the y-PUFs the algorithm can converge to the correct or flipped delay model and hence the overall convergence rate of the y-APUFs is higher than that of the x-PUFs when the correlation is similar. Nevertheless, there are some x-APUF instances that are easier to learn that some y-APUF instances (e.g. the second x-PUF). The smallest correlation in Figure 3b is 0.0461 for the third x-APUF instance which resulted in the smallest convergence rate of only 2 out of 1000. Hence, the convergence rate can become very small, but it is still higher than zero. Therefore, unlike claimed in [NSJ + 19], the iPUF can be attacked using a reliability based modeling attack. However, compared to the 8-XOR PUF, the (4,4)-iPUF has a much smaller correlation which also results in an overall smaller convergence rate of 831/1000 for (4,4)-iPUF compared to 981/1000 for 8-XOR APUF. Hence, attacking the iPUF using a reliability attack is harder than attacking an XOR PUF. Note that the correlation and hence the modeling resistance for each PUF instance is different. Therefore, testing only a single instance can be misleading. Another point to keep in mind is the fact that hardware implementations can deviate from the assumed i.i.d. distribution of APUF weights and might show a different behavior to a certain degree (for more discussion regarding this issue, see Section 6.6). The conclusion from our analysis is that the iPUF is insecure, but it can take a lot of simulation time and a lot of CRPs to attack some iPUF instances using a CMA-ES attack. If one can ensure that the modeling algorithm does not converge to already discovered PUF instances, the attack would be greatly improved.

A Differentiable Model for the Interpose PUF
Before developing an efficient reliability attack that can break large iPUF instances with a high success rate, we first deal with a prerequisite, namely building a differentiable model of the iPUF. This model can be used in an LR attack on the iPUF. More importantly, it serves as a building block for our full reliability attack.
Looking at the structure of the iPUF as described in Section 2.2, there seems to be an obstacle in the way of formulating a differentiable model for iPUF. The binary response r x of the x-XOR PUF is inserted into the challenge c to create the challenge c y . Computing a binary response from continuous delay values requires the discontinuous sign function which is unsuitable for gradient-based optimization 5 . However, we show that it is not necessary to use the binary r x but that its continuous equivalent d x is sufficient to build a differentiable model. As a preliminary step for the full model description, we first note that the computation of the feature vector φ y can be done on the basis of the feature vector φ x and the binary response r x : Now suppose you have the continuous response d x : Based on the observation the sign of −d x is the same as that of (−1) rx , one might be led to a naive computation of φ y : This formulation of φˆ, however, is incorrect as it does not allow for correct computation of r y . Suppose only a single APUF, characterized by w y is used in the y-component.
One half of terms of the scalar product φˆyw y is scaled by |d x |. The magnitude of d x varies from challenge to challenge and therefore no w y can be found that consistently leads to the correct response r y for all challenges. However, there exists an easy fix for the problem. The second half of φ y can simply also be multiplied by |d x |: That way, the sign of φ y w y and thereby the response r y can be computed correctly, for all applied challenges and independently of the magnitude of |d x |: Taking the absolute value is a continuous operation and can be used in models that are optimized by gradient descent 6 . The complete model for the LR attack on the iPUF, model IPUF W x,W y , which computes a continuous (probability) response r for a challenge c, is as given below:

Constraints and Combined Objectives -Streamlining the Reliability Attack
In Section 3, we showed that reliability attacks are harder to perform on the iPUF than on classical XOR PUFs. Attacking large iPUF instances with a regular CMA-ES reliability attack requires large training sets and many repeated trials. We use this as a motivation to develop a more efficient approach. The first main idea is to prevent repeated convergence of the reliability attack to the same APUF. This can be done by starting the reliability attack with multiple APUF candidates at the same time while penalizing their similarity. Thus, the optimization for each APUF candidate follows two objectives: Achieve a high correlation with the global PUF reliability and stay away from the other candidates. Given this formulation, it seems logical to include another target, that all candidates should achieve together: Compute the correct PUF response. This way, reliability and PUF response information are used at the same time.
This abstract goal of combining different targets at the same time needs to be translated into a practical optimization problem. Normally, the goal of optimization algorithms is to find a minimum (or maximum, respectively) of a function f (x) → R. For example, in gradient descent, f measures the loss, while in CMA-ES it is a measure of fitness. In our case, we now have multiple functions f 1 , . . . , f n that we want to optimize at the same time. The easiest way to do this, is to aggregate them into a single function: f total = f 1 + · · · + f n . This function f total can now be optimized with one of the readily available algorithms for real-valued functions. One problem, however, is that the individual functions can have different scales which could lead to one function dominating their sum. Thus, it makes sense to introduce hyper-parameter 1 , . . . , n that weight the importance of each target: In the following, we describe step-by-step our formulation of a combined attack on the XOR Arbiter PUF and the iPUF. The first step is the introduction of weight constraints to reliability attacks which is then combined with the direct-modeling objective. We describe this attack in the framework of gradient descent, thus speaking of losses and the inclusion of the LR objective. In theory, the same approach of combined objectives can also be followed in CMA-ES. However, our preliminary experiments on reliability attacks and the clear prevalence of gradient descent in direct-modeling attacks, lead to our choice of optimization algorithm. In Section 6.2, we show experiments that compare the algorithms in the original reliability attack setting to further motivate this choice.

Learning with Weight Constraints
As described in Section 3.2, each APUF has a different probability to be learned, depending on its correlation with the PUF output reliability. Large PUF instance with many individual APUFs are therefore hard to learn. To mitigate this, it is possible to simply add a term to the objective function which measures the similarity of the candidates for the single APUFs and incurs a penalty for a high degree of similarity. One possible way to measure the similarity is the Pearson correlation coefficient which we use in our implementation 8 . Typically, the weights of an APUF are modeled as i.i.d. random normal variables with zero mean. With increasing dimensionality, it rapidly becomes improbable that any two such random vectors are strongly linearly correlated. In Figure 4, we provide a sampled distribution for vectors with 64 elements. The largest correlation is far from the maximum of 1.0 or −1.0, respectively. However, it should also be clear that not all correlations are tightly packed around zero. It is also possible that manufactured PUFs show larger correlations than the idealized simulation model 9 . Therefore, one should not enforce strict orthogonality between vectors but one has to find a balance between the optimization goals of high correlation between model and PUF output and low correlation between APUF weight candidates.
In general, there are two main approaches to incorporating constraints into an attack. First, one can keep the iterative approach of the original CMA-ES attack. This way, one adds a new APUF weight candidate to a set in each iteration. The new candidate is constrained to be dissimilar to each previously attained candidate. The second approach is to compute all candidates at once, enforcing pair-wise dissimilarity simultaneously. In our actual implementation, we follow the second approach which allows us to combine it with a direct-modeling approach.
7 The use of a weighted sum is not the only option for multi-target optimization. For example, there is a line of work that targets the broader goal of finding sets of Pareto optimal solution, which can be used to explore different trade-offs between objectives (cf. [CLvV07,IHR07] in the context of evolution strategies). This more general approach is, however, not necessary for our attack, as we are only interested in finding a single optimum.
8 One could also use cosine similarity, which is mathematically similar to Pearson correlation, but we do not expect there to be a large difference for the attack performance.
9 Strong correlations between manufactured APUFs would reduce the unpredictability, which is a security issue in itself. The expected correlation between pairs is rather low which can be used as an argument for constraining APUF candidates in a reliability attack to have low correlation. However, the constraint should not be enforced too strictly, which would be detrimental as the true correlation has some expected deviation from zero.

Gradient-based Reliability Attacks with Constraints
In Section 2.3.2, we briefly summarized the reliability attack as presented by Becker [Bec15]. In this section, we show that CMA-ES can be replaced with a gradient-based optimization and more importantly how to constraint the optimization goal such that all candidate APUFs can be found iteratively.
The general framework of gradient-based optimization was described in Section 2.3.1 in the context of the LR direct-modeling attack. In order to move from the direct-modeling to the reliability domain, a loss function different from binary cross-entropy is required. Analog to the CMA-ES setting, we define a loss function based on Pearson correlation: Next assume, we are given a training set with challenge matrix C and corresponding feature vector matrix Φ and a reliability information vector h whose elements are computed as in Equation 13. Our model consists of only a single APUF instance and is equal to the unreliability measure as defined in Equation 14: model Arbiter This model does differ from the one defined for the original CMA-ES attack, as given in Equation 12, which would not be useable in gradient-based optimization due its inclusion of the step() function. The threshold parameter is not required anymore. With this, the total loss function loss total can be defined: Note that the larger the output of model w is, the more reliable the response is assumed to be. An output closer to zero indicates lower reliability, as the delay is closer to decision threshold at which the response bit flips from 0 to 1 and vice versa. The same holds true for reliability values computed according to Equation 13. As the loss function in gradient descent is canonically set up to be minimized, the above sum has to have multiplicative factor of −1. If this loss function is optimized repeatedly, one receives different candidates depending on the random initialization of w. In the next step, we want to transform this non-deterministic process into an iterative process.
Suppose, you already have a number of candidates for the weight w that presumably belong to the different APUF instances within the attacked XOR or iPUF instance. Let these candidates be the rows of matrix W cand . The idea now is to add a constraints term to loss total that discourages the convergence of the next learned weight w to already  Equation 27, the model reliability output is invariant to multiplying w by −1. Therefore, one has to prevent both positive and negative correlation between W cand and w. The constant a enables balancing between both optimization goals and has to be adjusted, depending on the attacked PUF and the size of the set of candidates.

Combining Reliability Attack and Logistic Regression
In practice, we found it to be better to change the approach from learning one candidate APUF weight at a time to learning all weights at the same time. In this case, the APUF weights are not constrained by a fixed set of reference weights. Instead, all APUF weights are prevented from forming strong pair-wise correlations. This is computationally more efficient, as parallelism can be exploited. To further optimize the learning problem, we included a loss term for the actual response, thereby combining direct LR modeling with a reliability attack.
Suppose now, you are given a feature vector matrix Φ and the corresponding reliability vector h and PUF response vector r. The goal is to find W which contains the weights of all k APUFs. Two model functions, model total and model Arbiter , are required. The first describes the complete PUF and outputs a response probability and the second is equal to Equation 27, returning the reliability measure of a single APUF. Then, one can set up a loss function that encompasses reliability and LR: The first term gives an incentive for better prediction accuracy, while the second rewards correlation between the individual APUFs and the output reliability. The third term discourages similarity between the individual APUFs. If this term is not included, it is possible that multiple APUFs converge to the same weight vector. The constants a and b have to be adjusted to each attack scenario 11 .
The combined loss as shown above can be directly applied to learning XOR-APUFs. The iPUF, however, requires slight technical modifications that stem from the fact that APUFs from the x-and y-XOR PUFs are not interchangeable and the ambiguity of the y-APUFs that was mentioned in Section 3.1.

Combined Attack on Interpose PUFs
Based on the analysis in Section 3.1, it is clear that one can generally apply the reliability attack on iPUFs and find candidates for the x-APUFs and reduced-form candidates for the y-APUFs. However, as previously noted there is ambiguity regarding the y-APUF weights. Inverting one half of a y-APUF weight vector does not change its reliability loss.
When you learn all y-weights iteratively, it is therefore possible that you end up with a set of weights of which only some are partially inverted. Inserting this set into an iPUF model would not lead to high accuracy. Instead, one would have to devise a search strategy that finds a set of coherent y-weights. Learning all y-weights at the same time, while also optimizing prediction accuracy, as described in the previous section solves this problem. However, one still has to amend the constraints term that prevents convergence to similar weights. This term now has to be split into two, one of which discourages similarity between the first half of all APUF weight candidates and the other acting on the second half of all APUF weight candidates. The correlation coefficient over complete vectors cannot distinguish between a vector and its partially inverted copy which would lead to ineffectively constrained y-weights. Another issue that can come up, is the mixing of x and y-APUF candidates. The regular reliability loss cannot readily differentiate between the x-APUFs and y-APUFs which can lead to weight candidates ending up in the wrong XOR PUF which prevents successful learning. To combat this, we extended the reliability loss term and added extra loss if x-stage candidates show high reliability correlation if their weight vector is partially inverted. Conversely, the y-stage candidates are rewarded in terms of loss when they show high reliability correlation if their weight vector is partially inverted. We give the complete loss function as a reference in Appendix A.1. With this loss function it is possible to directly learn an iPUF based on response and reliability information. We used this single-pass approach to successfully learn up to (1,10) and (4,4)-iPUFs.

Multi-pass Attack on Interpose PUFs
When scaling up the number of x-PUFs we came upon the same observation that had been previously made by Wisiol et al. [WMP + 20] and was leveraged by them into a directmodeling attack, namely that the y-PUFs can be learned without learning the x-PUFs. This is possible because for a subset of all challenges, the x-response has no effect on the overall PUF response. This subset is informative enough for successfully learning y-APUFs. Instead of further optimizing the single-pass approach for larger numbers of x-PUFs, we followed an approach that is reminiscent of the attack by Wisiol  3. Use the learned weights from the second step as x-PUF candidates and the y-weights from the first stage to further optimize loss IPUF combined . After the first step, the y-APUF weights are already modeled very well with a correlation coefficient of more than 0.99 to their respective PUF weights. The x-APUFs, however, have on average a weaker reliability correlation signal and are less likely to be found and, despite constraints, tended to mistakenly converge to y-APUFs. Increasing the constraint loss in the first step did not help convergence in our experiments but showed rather quickly detrimental effects on learning of the y-weights. In the second step, the adapted constraints setup is more helpful. Recall that in the first step, the weights are constrained to not become too similar to each other. From the point of view of a single APUF candidate, the set of weights to which it is supposed to be dissimilar changes every optimization step. This is quite different to the second step, in which the already learned y-weights are used as fixed constraints. These provide a stable feedback and we found them to be more important than the hypothetical x-responses which usually had a rather high mismatch with the actual x-responses. In our experiments, the second step usually provided x-APUFs with a much lower correlation to their model weights, compared to their first step y-APUF counterparts. However, these x-APUF weights turned out to be good enough as a starting point to resume the optimization of all weights in the third step.
We want to note that the description of the multi-pass approach resembles our actual implementation and shows its experimentally-driven evolution. The exact details such as whether to use iPUF or XOR PUF models in the first and second step provide some flexibility. The main point is that learning a set of coherent y-weights first can enable more efficient learning of the x-weights.

Simulation Results and Analysis
In this section, we show various simulation-based attack results. Of these results, the two main points we make are that the combined optimization objectives perform better than the classical reliability attack, and that the iPUF can be attack be attacked with much smaller training set sizes than those required in classic non-reliability attacks. Additionally, we discuss the applicability of our attack to actual hardware implementations.
We have followed an approach on simulated PUF attacks as stated in previous publications like [RSS + 10, TB15] by drawing Arbiter stage delays from an i.i.d. normal distribution with a variance of one. Based on the Arbiter PUF delay model, we generate (noisy) challenge-response pairs for the training and test sets. The attacker model follows the description in Section 2.3, i.e., a training set of challenge-response pairs is collected while the attacker has time-limited direct access to the PUF, which is used to build a model that is supposed to pass future authentication attempts. All APUFs in our simulations have 64 stages 13 . In the following, we state the variance of the APUF noise σ 2 noise where necessary. The model accuracy was evaluated on a test set, independent from the training set, of 5, 000 CRPs in all simulations. The PUF accuracy was estimated by computing the average match between noise-free responses and noisy responses. The model accuracy was computed between noisy responses and the model predictions. We used an Intel Xeon E5-2650 v3 CPU clocked at 2.30GHz for our attack simulations. Many experiments ran single-threaded and the number of threads is stated where this was not the case.

Notes on Attack Implementation
The popularity of deep-learning has lead to very powerful software frameworks, such as Tensorflow or Pytorch, that support automatic gradient computation. These accessible tools make it much easier to experiment with the kind of modeling approaches that we propose in this work. We chose Pytorch [PGM + 19] for our implementation 14 . We used Adadelta and ADAM as optimization routines with a default batch size of 256 and did not find them to be different from each other in performance. For the experiments in Section 6.2, we used a publicly available implementation 15 of CMA-ES.
In general, PUF models are much smaller in parameter size than neural networks which leads, together with small batch sizes, to little advantage for parallelization which suffers from too much overhead. Due to this reason, we performed all computations on CPU and did not use GPUs. To mitigate this circumstance, however, we implemented parallelization at the level of trials. As the chance of success for an attack depends on the random initialization of the model, it is often required to run multiple trials for a single PUF instance. One can simply initialize multiple models and optimize them in the same loop over the same training set. This is the reason why we give the run time for our attack for a number or parallel trials in the next sections.
The combination of different optimization goals, as described in the previous sections, requires the attacker to define appropriate constants that scale the different optimization terms. The general rule is that increasing the loss of one of the terms scales its importance up in comparison to the other terms. We do not state exact guidelines on how to set these constants, as we resorted to manual optimization. First, we disabled LR and weight constraints and observed how well individual APUF instances were found. If this unconstrained reliability attack did not converge to actual APUF instances, we increased the training set size. Once convergence did happen at all, we enabled weight constraints and scaled their loss up, observing whether the set of APUF candidates started converging to a more diverse set of APUF instances. Once achieved, we enabled the LR term, scaling up its loss until all individual APUFs could be found.
A good choice for the value of the constants depends both on PUF parameters, training set size, noise level and the numerical implementation of the attack itself. We give our choices for the constants for the experiments of Section 6.3 in Appendix A.2 as a reference but do not consider them optimal. We assume that further fine-tuning, especially in the case of the multi-pass approach, can lead to better success rates and higher model accuracy.

Comparison between CMA-ES and Gradient Descent
In this section, we show that CMA-ES and (stochastic) gradient descent can both be used to solve the optimization problem posed in the original reliability attack: Find APUF candidates whose reliability correlates well with the global PUF reliability. To this end, we simulated populations with 50 members for 4-XOR, 6-XOR, 8-XOR and 10-XOR PUFs and attacked each PUF instance using both optimization algorithms. The error variance σ 2 was set to 0.1 for all PUFs. For CMA-ES, we set the maximal number of generations to 300. We left other hyper-parameters at default values [HO01], in particular, λ and µ (see Section 2.3.2) : λ = 8 + 3ln(n) = 20, where n is the search space dimensionality, and µ = λ 2 = 10. For gradient descent, we used the Adadelta optimizer and limited the number of epochs to 25. For each PUF, we collected 16 APUF candidates. For the whole population, we then computed the convergence rate, i.e., the proportion of APUF candidates that show a Pearson correlation of at least 0.98 with any of the actual APUFs. The results can be found in Figure 5. It can be seen that both optimization approaches find solutions with approximately the same training set size. However, the convergence rate for gradient descent is generally higher and approaches its maximum while the convergence rate for CMA-ES seems to level off sooner.
In addition to the better convergence rate, gradient descent also ran about twice as fast. This difference is, however, implementation dependent. We did not implement early stopping, therefore the algorithms always used the maximum number of generations and epochs, respectively. For example, in the case of the 10-XOR PUF, the average number of generations required for a successful CMA-ES trial ranged from 250 at 25, 000 CRPs to 126 at 400, 000 CRPs. In contrast, for gradient descent, 12 epochs were on average required at 25, 000 CRPs, and 1.4 epochs at 400, 000 CRPs, respectively. Despite the difference in the optimization algorithms, comparing the number of generations and epochs can give an indication of how many passes through the training are required. This highlights one difference in randomly sampling weight updates and basing weight updates on gradient information.
Previously In this experiment, we compare the convergence rate between CMA-ES and (stochastic) gradient descent for the basic reliability attack, in which a single APUF candidate is found in each trial. A PUF population size of 50 was used. The error variance σ 2 was set to 0.1 and reliability information was collected over 10 repetitions.

Comparison of Optimization Objectives
In Section 5.3, we described how multiple optimization objectives such as single APUF reliability correlation and LR can be combined. In this section, we show that this approach is more efficient than a regular reliability attack. To this end, we performed three attacks on two iPUF types, (5,5)-and (1,10)-PUFs. For each type we created a population of 20 PUFs. The same training set of challenges, responses, and reliability values was used in all three attacks for each individual PUF. The first attack mirrors the original reliability attack, in which APUFs are targeted individually without constraints. In the second attack, we modeled all individual APUFs at the same time and used constraints to prevent them from converging to the same weights. In the third attack, we additionally included the LR objective and used the single-pass approach for (1,10)-PUF and the multi-pass approach for the (5,5)-PUF. In Figure 6, we show how many individual APUFs for each PUF instance were found by the different approaches. It can be seen that learning with constraints outperforms the regular reliability-attack, while both are clearly beaten by the optimized combined-approach. We also attacked the same (5,5)-PUF instances with a CMA-ES reliability attack but only had a two convergences in 12k trials which shows that for such a hard PUF instance, gradient descent outperforms CMA-ES. In each experiment, 20 PUFs were attacked with eight trials in parallel in which the best trial was chosen for the attacks with constrains and 8(x + y) trials for the unconstrained reliability attack which targets a single PUF model. An APUF is considered to be found when the Pearson correlation between PUF and model is larger than 0.99. The (5,5)-iPUF was learned with 200,000 unique challenges, (1,10) with 500,000. In both cases σ 2 was set to 0.1 and reliability information was collected over 10 repetitions.

Scaling of Reliability-Attacks
The previous results already show that the combined attack can break large iPUF instances. To give a better overview of how the attack scales, we ran experiments for many instance sizes for both the regular XOR PUF and the iPUF. We generally kept the same approach of creating 20 PUFs for each instance size and running multiple trials for each individual PUF.
The first point we want to make is the fact that reliability attacks require much less training data than LR attacks, not only for XOR-Arbiter PUFs but also for the iPUF. To this end, we perform a comparison of the required training set size between the split PUF technique by Wisiol et al.[WMP + 20] and our attack on the (1,X)-iPUF. We did not implement the split PUF technique ourselves but rather give a lower bound of its complexity by performing the regular LR attack on X-XOR Arbiter PUFs instead. This approach is motivated by the analysis of Wisiol et al.[WMP + 20] that their attack essentially reduces the security of an (X,Y)-iPUF to that of a max(X,Y)-XOR Arbiter PUF. In a first step, we computed the success rate of both attacks for a large range of training set sizes for differently-sized PUF instances. For the reliability attack, we fixed the noise level σ 2 noise of a single APUF to 0.1 and varied the size of the lower layer to up to fourteen APUFs. The largest LR attacked featured an 8-XOR Arbiter PUF. The resulting learning curves can be found in Figures 7a and 7b. These figures already give an impression of the difference in scaling between the attacks which becomes more obvious if one compares the required training set for a specific success rate over the different instance sizes. We provide this in Figure 7c for the lowest success rate greater than zero that we found in our experiments. Additionally we show the same result for our LR attack on the iPUF and results given by Wisiol et al.[WMP + 20] for their split PUF technique. The curves for the latter two attacks show the training set for success rates significantly higher than zero but are still informative in this context. In case of all LR-based attacks, one can clearly see the exponential scaling of the training set size while the reliability attack only shows linear scaling. This enables attacks on even larger instances such as the (1,14)-iPUF with training set sizes that are orders of magnitude lower. Note that here the training set size denotes the number of unique challenges and does not include the number of PUF queries that are required to compute reliability values. The number of repeated queries, however, does not grow with larger PUF instances. Still, the question remains which impact this number has on the success rate of the attack. To answer this, we conducted an experiment in which we varied the number of queries in the range from 2 to 25. The results are shown in Figure 8. It can be seen that more repetitions quickly provide only diminishing returns and that the success rate levels off between 5 and 10 repetitions. For consistency reasons, we set the number of repetitions to 10 in all other experiments.
To give further insight into the performance of our reliability attack we give further details in tabular form. First, we show the performance for regular XOR Arbiter PUFs in Table 2. This result is very much in line with previous publications (cf. [Bec15]) and also shows that reliability attacks on XOR Arbiter PUFs can be modeled with a training set that grows linearly with respect to the number of APUFs k. The runtime for the attack on 10-XOR PUFs for twelve parallel trials is below two hours at a training set size of 200,000, which is very much practical. This duration could also be further improved, for example, by implementing early stopping. In an actual attack, one only cares about finding a single fitting solution, while we let run all trials for the complete number of epochs due to implementation choices and to compute success rates. Next, in Table 3, we give results for our single-pass attack on (1,x)-iPUFs and for our multi-pass attack on (x,x)-iPUFs. As previously seen, the attacks scales well and we show in particular that (1,10)-iPUFs, which were proposed as potentially secure configuration by Nguyen et al. [NSJ + 19], can be broken with less than 10 7 PUF queries. In comparison, based on numbers by Tobisch and Becker [TB15] or our experiments in Figure 7a, one can estimate that a 10-XOR APUF alone would require several hundred million CRPs in the LR setting. One can also set result for the 10-XOR PUF from Table 2 in relation with results for the (1,10)-iPUF and the (5,5)-iPUF, which are close in terms of the number of APUFs. The attacks on these share a reasonable runtime at the same order of magnitude of around two hours or less, showing little advantage for the iPUF in this regard. Note, however, that the number of CRPs, trials and epochs differ, which govern the success rate and runtime.   Table 6. For the experiments in (b), we have fixed the noise σ 2 noise per single APUF to 0.1 and the number or reliability queries per challenge to 10 for all instances. All other experiments were run noise-free. In (c), we compare the training size complexity for different attacks. The curves for the LR attack on XOR-Arbiter PUFs and the reliability attack on iPUFs are taken from the data presented in (a) and (b) and show the lowest training set size for which the success rate was above 0. The LR attack on the iPUF is given for a higher success rate (see Table 4). The values for the LR-based split attack are as given by Wisiol et al.[WMP + 20]. The success rate for these values was one, except for the (1,9)-iPUF where it is 0.26. Note that the training set size in all three figures denotes the number of unique challenges in the training set which does not include the additional PUF queries that are required for the reliability attack. The number of additional queries does not grow with the number of single APUFs. In general, we found that the multi-pass attack on (x,x)-iPUFs is feasible but tuning the constraint constants is more cumbersome. Increasing the number of x-XOR APUFs makes the attack harder, though it is hard to say to which degree and how it compares to increasing the number of y-XOR PUFs. We conclude that this attack could be more streamlined and investigated further in future work.

Results for the LR Attack on iPUFs
In Section 4, we showed that a differentiable model of the iPUF exist. We use this as one element of our combined reliability attack but it can also be used in a direct-modeling LR attack without reliability information. As shown in Table 4, we applied this adapted LR attack on instances up to (1,5)-and (4,4)-iPUFs with moderate CRP sizes. For larger instances, learning became less successful. Usually, the y-PUF could still be learned but the x-PUF remained rather random, leading to an accuracy distinctly above 0.5 but much below the targeted accuracy of above 0.95 17 . We did not investigate further improvements of this attack, given the fact that Wisiol    One important component of our attack is the introduction of a similarity constraint that punishes APUF candidates that converge to the same target. Therefore, the question arises whether the success of our attack might be impeded if actual PUFs in hardware do show a higher degree of correlation than the simulation model. For the lack of data from PUF implementations, we cannot conclusively answer this question. However, we believe that such correlations do not prevent reliability attacks. The first reason for this is that our attack has to deal with randomly occurring correlations in any case. In Figure 4, we showed that even if weights are distributed identically and independently, you can expect that some amount APUF pairs will exhibit correlations. As a consequence, we do not force the correlations between APUF candidates to be zero. Instead, a constraint constant provides a trade-off which can be relaxed, if required. Additionally, the explicit dissimilarity constraint is not the only deciding factor in our attack. The inclusion of the LR objective is another incentive for the APUF candidates to converge to different targets, that jointly model the target PUF.
Furthermore, we want to note that i.i.d. distributed weights are the ideal case for implementations that PUF designers strive for. Strong correlations in a PUF implementation have to be avoided and could not be considered a countermeasure. Side-effects of violating the i.i.d. assumption include worsened uniformity and better predictability (cf. results by Nguyen et al. [NSJ + 19], which show that the success of a reliability attack can be implementation-dependent).
We regard it as important future work to examine the correlations for actual hardware implementations and to validate our attack approach in this setting. Ideally, this would include a range of different implementations, including ASIC results, which we assume to be quite different from FPGAs.

Conclusion
In this work, we showed how reliability attacks on APUF-based Strong PUFs can be considerably improved by the addition of constraints and the combination of different optimization goals. Gradient-based optimization provides a flexible framework for the applications of these ideas and powerful software packages such as Pytorch of Tensorflow ease the implementation. The inclusion of multiple objectives, however, introduces a need for weighing these objectives. Overconstraining the solution by giving too much weight to one of the objectives can limit model accuracy or overall prevent successful learning. Therefore, our solution does require the tuning of constraint constants whose optimum is different for differently instantiated PUFs.
We analyzed the iPUF in regard to susceptibility to reliability-attacks. The original CMA-ES-based approach is not entirely disabled by the iPUF design, unlike its original security analysis suggested. The probability of finding a specific APUF depends on its reliability correlation magnitude in relation to the other APUFs. Overall, the probability of finding all APUFs is greater than zero and a CMA-ES attack can be conducted, if the training set is large and enough computational time is spent. The application of our novel multi-objective approach in the gradient descent framework increases the attack efficiency dramatically in comparison to the original reliability attack. This allows us to break large instances like the (1,10)-iPUF, which were previously thought to be secure.
We note that the iPUF design does make reliability-attacks harder to perform and increases the required size of the training set, compared to similarly parametrized XOR PUFs. The reason for this, however, does not so much lie in the difference between the reliability correlation magnitudes for x-XOR PUF and y-XOR PUF which was thought to be critical by Nguyen et al. [NSJ + 19]. Instead, the advantage of the iPUF lies in the overall decrease in the reliability correlation magnitude. We believe that this advantage is not big enough to make secure iPUF implementations feasible in practice.
Our methodology of measuring correlation magnitudes, can be used by PUF designers to gauge the resistance against reliability attacks before actually performing any concrete attacks. Ideally, one would find a construction that requires an exponentially growing training set size to achieve high correlation magnitudes. An open question is to what degree asymmetric designs such as the iPUF in which APUFs contribute differently are a viable to path to more secure constructions. As demonstrated in our work and in the attack by Wisiol et al. [WMP + 20], modeling attacks are very flexible and can target individual components, effectively reducing the security level to that of the most-resilient single component. We hope that our presented multi-objective attack framework is helpful to other researchers. In future work, it might be desirable to further streamline the process of setting up a successful attack. This includes, for example, finding good constraint constants in an automatic fashion rather than by a manual search.

A.1 Combined Attack Implementation Details
Assuming that an Interpose PUF is characterized by weight matrices W x and W y , one can define matrices W x , W y-red and W y-red based on Equations 15 and 16. Based on these weights one can create a loss function for the iPUF that includes all relevant loss terms that are explained in Section 5.
Note that we have added a multiplicative constraint to each term even though one less constant would be sufficient. We found the full set of constants helpful for experimentation and manual setting.
The complexity of some constraint terms scales quadratically in the number of APUFs which in practice does not lead to unreasonable computation times if the loss function is implemented efficiently using vectorization. Parallelization, especially at the level of trials, helps to further speed up attacks.

A.2 Constraint Constants
Here, we provide our settings for the constraint constants that we used in the experiments whose results are given in Section 6.3. all instances 0.5 12.00 1.00 0.20 0.00 Table 6: Constraints for single-pass experiments in Table 3 and the combined reliability attack in Figure 7.