<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE article PUBLIC "-//NLM//DTD Journal Publishing DTD v2.3 20070202//EN" "journalpublishing.dtd">
<article article-type="research-article" dtd-version="2.3" xml:lang="EN" xmlns:mml="http://www.w3.org/1998/Math/MathML" xmlns:xlink="http://www.w3.org/1999/xlink">
<front>
<journal-meta>
<journal-id journal-id-type="publisher-id">Front. Mater.</journal-id>
<journal-title>Frontiers in Materials</journal-title>
<abbrev-journal-title abbrev-type="pubmed">Front. Mater.</abbrev-journal-title>
<issn pub-type="epub">2296-8016</issn>
<publisher>
<publisher-name>Frontiers Media S.A.</publisher-name>
</publisher>
</journal-meta>
<article-meta>
<article-id pub-id-type="publisher-id">851085</article-id>
<article-id pub-id-type="doi">10.3389/fmats.2022.851085</article-id>
<article-categories>
<subj-group subj-group-type="heading">
<subject>Materials</subject>
<subj-group>
<subject>Original Research</subject>
</subj-group>
</subj-group>
</article-categories>
<title-group>
<article-title>Development of a Robust CNN Model for Capturing Microstructure-Property Linkages and Building Property Closures Supporting Material Design</article-title>
<alt-title alt-title-type="left-running-head">Mann and Kalidindi</alt-title>
<alt-title alt-title-type="right-running-head">Robust CNN Supporting Material Design</alt-title>
</title-group>
<contrib-group>
<contrib contrib-type="author">
<name>
<surname>Mann</surname>
<given-names>Andrew</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<uri xlink:href="https://loop.frontiersin.org/people/1620916/overview"/>
</contrib>
<contrib contrib-type="author" corresp="yes">
<name>
<surname>Kalidindi</surname>
<given-names>Surya R.</given-names>
</name>
<xref ref-type="aff" rid="aff1">
<sup>1</sup>
</xref>
<xref ref-type="aff" rid="aff2">
<sup>2</sup>
</xref>
<xref ref-type="corresp" rid="c001">&#x2a;</xref>
<uri xlink:href="https://loop.frontiersin.org/people/585767/overview"/>
</contrib>
</contrib-group>
<aff id="aff1">
<sup>1</sup>
<institution>Georgia Institute of Technology</institution>, <institution>School of Materials Science and Engineering</institution>, <addr-line>Atlanta</addr-line>, <addr-line>GA</addr-line>, <country>United&#x20;States</country>
</aff>
<aff id="aff2">
<sup>2</sup>
<institution>Georgia Institute of Technology</institution>, <institution>George W. Woodruff School of Mechanical Engineering</institution>, <addr-line>Atlanta</addr-line>, <addr-line>GA</addr-line>, <country>United&#x20;States</country>
</aff>
<author-notes>
<fn fn-type="edited-by">
<p>
<bold>Edited by:</bold> <ext-link ext-link-type="uri" xlink:href="https://loop.frontiersin.org/people/174214/overview">Roberto Brighenti</ext-link>, University of Parma, Italy</p>
</fn>
<fn fn-type="edited-by">
<p>
<bold>Reviewed by:</bold> <ext-link ext-link-type="uri" xlink:href="https://loop.frontiersin.org/people/1653668/overview">Niaz Abdolrahim</ext-link>, University of Rochester, United&#x20;States</p>
<p>
<ext-link ext-link-type="uri" xlink:href="https://loop.frontiersin.org/people/1303229/overview">Hamid Akbarzadeh</ext-link>, McGill University, Canada</p>
</fn>
<corresp id="c001">&#x2a;Correspondence: Surya R. Kalidindi, <email>surya.kalidindi@me.gatech.edu</email>
</corresp>
<fn fn-type="other">
<p>This article was submitted to Computational Materials Science, a section of the journal Frontiers in Materials</p>
</fn>
</author-notes>
<pub-date pub-type="epub">
<day>11</day>
<month>03</month>
<year>2022</year>
</pub-date>
<pub-date pub-type="collection">
<year>2022</year>
</pub-date>
<volume>9</volume>
<elocation-id>851085</elocation-id>
<history>
<date date-type="received">
<day>09</day>
<month>01</month>
<year>2022</year>
</date>
<date date-type="accepted">
<day>23</day>
<month>02</month>
<year>2022</year>
</date>
</history>
<permissions>
<copyright-statement>Copyright &#xa9; 2022 Mann and Kalidindi.</copyright-statement>
<copyright-year>2022</copyright-year>
<copyright-holder>Mann and Kalidindi</copyright-holder>
<license xlink:href="http://creativecommons.org/licenses/by/4.0/">
<p>This is an open-access article distributed under the terms of the Creative Commons Attribution License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the original publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these&#x20;terms.</p>
</license>
</permissions>
<abstract>
<p>Recent works have demonstrated the viability of convolutional neural networks (CNN) for capturing the highly non-linear microstructure-property linkages in high contrast composite material systems. In this work, we develop a new CNN architecture that utilizes a drastically reduced number of trainable parameters for building these linkages, compared to the benchmarks in current literature. This is accomplished by creating CNN architectures that completely avoid the use of fully connected layers, while using the 2-point spatial correlations of the microstructure as the input to the CNN. In addition to increased robustness (because of the much smaller number of trainable parameters), the CNN models developed in this work facilitate the construction of property closures at very low computational cost. This is because it allows for easy exploration of the space of valid 2-point spatial correlations, which is known to be a convex hull. Consequently, one can generate new sets of valid 2-point spatial correlations from previously available valid sets of 2-point spatial correlations, simply as convex combinations. This work demonstrates the significant benefits of utilizing 2-point spatial correlations as the input to the CNN, in place of the voxelated discrete microstructures used in current benchmarks.</p>
</abstract>
<kwd-group>
<kwd>convolutional neural networks</kwd>
<kwd>property closures</kwd>
<kwd>2-point spatial correlations</kwd>
<kwd>convex hull</kwd>
<kwd>microstructure design</kwd>
</kwd-group>
</article-meta>
</front>
<body>
<sec id="s1">
<title>1 Introduction</title>
<p>The microstructure<xref ref-type="fn" rid="fn1">
<sup>1</sup>
</xref> of a material has a causal relationship with its effective anisotropic properties. Therefore, it should be theoretically possible to design the microstructure for optimal performance, which is typically specified in terms of a set of desired effective material properties. In practice, the microstructure-property relationships are most commonly explored using computationally expensive physics-based simulation tools (<xref ref-type="bibr" rid="B22">Ghosh et&#x20;al., 1995</xref>; <xref ref-type="bibr" rid="B31">Kalidindi and Schoenfeld, 2000</xref>; <xref ref-type="bibr" rid="B52">Roters et&#x20;al., 2010</xref>; <xref ref-type="bibr" rid="B58">Wargo et&#x20;al., 2012</xref>; <xref ref-type="bibr" rid="B5">Brands et&#x20;al., 2016</xref>). However, such computational tools allow exploration mainly in the forward direction, i.e.,&#x20;going from given microstructures to the estimation of their effective properties. Microstructure design can be achieved through iterative evaluations of the forward model to minimize a suitably defined objective function on the targeted properties. However, the high computational expense of the physics-based forward models poses a major hurdle for such design efforts.</p>
<p>Many efforts in prior literature have aimed to reduce the computational cost of the forward models described above, since inverse solutions often require the execution of a very large number of forward trials. Such efforts have explored both analytical approaches as well as emergent data-driven approaches. The most impactful efforts utilizing analytical approaches have been based on Kroner&#x2019;s perturbation expansion of the solution for the effective elastic stiffness of a composite material (<xref ref-type="bibr" rid="B38">Kr&#xf6;ner, 1971</xref>). In this formalism, the effective material property is expressed as a series whose terms systematically utilize increasingly higher-order spatial correlations (i.e.,&#x20;n-point spatial correlations) of the different material local states in the microstructure. In most practical applications, the series expansion is truncated to include up to 2-point spatial correlations (<xref ref-type="bibr" rid="B57">Torquato, 2002</xref>; <xref ref-type="bibr" rid="B1">Adams et&#x20;al., 2013</xref>; <xref ref-type="bibr" rid="B29">Kalidindi, 2015</xref>). An implicit benefit of these analytical approaches is that they permit formal application of optimization methods (many of which require computation of the gradients of the specified objective function with respect to microstructural variables) in solving microstructure design problems. Examples of these efforts can be found in the development and application of the Microstructure-Sensitive Design (MSD) framework (<xref ref-type="bibr" rid="B17">Fullwood et&#x20;al., 2007</xref>; <xref ref-type="bibr" rid="B16">Fullwood et&#x20;al., 2008a</xref>; <xref ref-type="bibr" rid="B18">Fullwood et&#x20;al., 2010</xref>; <xref ref-type="bibr" rid="B1">Adams et&#x20;al., 2013</xref>). Prior MSD efforts have demonstrated the viability of designing simple microstructures (e.g., composites with two isotropic phases, single phase polycrystalline materials) to meet designer specified target properties (<xref ref-type="bibr" rid="B2">Adams et&#x20;al., 2001</xref>; <xref ref-type="bibr" rid="B15">Fast et&#x20;al., 2008</xref>; <xref ref-type="bibr" rid="B35">Knezevic et&#x20;al., 2008</xref>; <xref ref-type="bibr" rid="B54">Shaffer et&#x20;al., 2010</xref>; <xref ref-type="bibr" rid="B1">Adams et&#x20;al., 2013</xref>). The application of the MSD framework has been largely confined to simple microstructures and simple physics due to the difficulties encountered in computing the convolution integrals involved in the series expansions. Some of the main hurdles encountered arise from the need to find Green&#x2019;s function solutions for the specific governing field equations and the reliable computation of the principal value (<xref ref-type="bibr" rid="B16">Fullwood et&#x20;al., 2008a</xref>; <xref ref-type="bibr" rid="B18">Fullwood et&#x20;al., 2010</xref>; <xref ref-type="bibr" rid="B1">Adams et&#x20;al., 2013</xref>). Furthermore, the perturbation series expansion has also been shown to be limited in practice to moderate contrast (refers to the degree to which the local properties can change from one location to another in the microstructure) material systems (<xref ref-type="bibr" rid="B28">Kalidindi et&#x20;al., 2006</xref>; <xref ref-type="bibr" rid="B18">Fullwood et&#x20;al., 2010</xref>). This limitation is due to the challenges encountered in achieving convergence in the series expansion with the systematic inclusion of higher-order spatial correlations (<xref ref-type="bibr" rid="B57">Torquato, 2002</xref>; <xref ref-type="bibr" rid="B16">Fullwood et&#x20;al., 2008a</xref>; <xref ref-type="bibr" rid="B18">Fullwood et&#x20;al., 2010</xref>).</p>
<p>Data-driven approaches have aimed to overcome the shortcomings of the analytical approaches described above by producing low-computational cost surrogates trained on the high-computational cost physics-based numerical models [e.g., representative volume elements modeled by finite element models (FEM)]. If these surrogates exhibit adequate accuracy, their low computational cost clearly justifies their use in microstructure design efforts. A prime example of these efforts can be seen in the Materials Knowledge System (MKS) (<xref ref-type="bibr" rid="B30">Kalidindi et&#x20;al., 2010</xref>; <xref ref-type="bibr" rid="B39">Landi et&#x20;al., 2010</xref>; <xref ref-type="bibr" rid="B29">Kalidindi, 2015</xref>; Brough et&#x20;al., 2017) framework, which employs a novel feature engineering approach for material microstructures by combining the formalism of the n-point spatial correlations mentioned above with machine learning tools such as the principal component analysis (PCA). The MKS framework learns the salient (low-dimensional) microstructure features in a completely unsupervised manner. These low-dimensional features are then used to build data-driven surrogate models for the reliable prediction of a broad range of material properties of interest. In typical MKS applications, these surrogate models are trained on datasets generated by physics-based numerical tools. The viability of the MKS approach has been demonstrated on a broad class of material structures and applications (<xref ref-type="bibr" rid="B11">Cecen et&#x20;al., 2014</xref>; Brough et&#x20;al., 2017; <xref ref-type="bibr" rid="B40">Latypov et&#x20;al., 2019</xref>). In recent extensions of the MKS framework (<xref ref-type="bibr" rid="B9">Cecen et&#x20;al., 2018</xref>; <xref ref-type="bibr" rid="B61">Yang et&#x20;al., 2018</xref>; <xref ref-type="bibr" rid="B13">Eidel, 2021</xref>), convolutional neural network (CNN) based surrogates have been explored, which bypass the feature engineering steps and build structure-property relationships directly from the input voxelated microstructure volumes. These CNN-based surrogates have demonstrated excellent accuracy, even for high contrast composites (<xref ref-type="bibr" rid="B61">Yang et&#x20;al., 2018</xref>; <xref ref-type="bibr" rid="B13">Eidel, 2021</xref>). Although the CNN-based models offer a highly accurate and low-computational cost tool to predict the effective property of a given microstructure, they encounter certain limitations that arise from the difficulty of incorporating known physical concepts into the CNN-based surrogate models. For example, when one imposes periodic boundary conditions on a representative volume element (RVE) of a microstructure, the predicted effective property exhibits translational invariance<xref ref-type="fn" rid="fn2">
<sup>2</sup>
</xref>. The most commonly used CNN architectures do not exhibit this characteristic implicitly. Most importantly, CNN-based models are prone to model over-fit due to their large number of tunable parameters. Despite these limitations, recent work has demonstrated the positive impact of data-driven methods on topology optimization (<xref ref-type="bibr" rid="B36">Kollmann et&#x20;al., 2020</xref>; <xref ref-type="bibr" rid="B62">Yilin et&#x20;al., 2021</xref>) and inverse design of microstructures (<xref ref-type="bibr" rid="B27">Jung et&#x20;al., 2020</xref>; <xref ref-type="bibr" rid="B55">Tan et&#x20;al., 2020</xref>).</p>
<p>This work aims to combine the advantages of both the analytical and data-driven approaches described above. First, this work employs the microstructure hull concept introduced in the MSD framework, which represents the complete space of physically realizable structures in a compact and convex space. Second, this work builds CNN-based surrogates using the 2-point spatial correlation maps as inputs, as opposed to using the voxelated microstructures directly. The approaches described in this work offer many advantages: 1) The use of 2-point spatial correlations as input to the CNN models automatically imparts translational invariance. 2) The change of the input to the CNN, from the voxelated microstructures (RVEs) to the 2-point spatial correlation maps, is expected to produce a more accurate and robust surrogate model (compared to current benchmarks) with a significantly smaller number of trained parameters (and the associated training cost). 3) The proposed strategy allows one to explore the complete space of possible property combinations in a highly practical manner (by limiting the exploration to the 2-point spatial correlations hull); this construct has been termed as property closure in prior work (<xref ref-type="bibr" rid="B49">Proust and Kalidindi, 2006</xref>; <xref ref-type="bibr" rid="B17">Fullwood et&#x20;al., 2007</xref>; <xref ref-type="bibr" rid="B59">Wu et&#x20;al., 2007</xref>; <xref ref-type="bibr" rid="B35">Knezevic et&#x20;al., 2008</xref>; <xref ref-type="bibr" rid="B18">Fullwood et&#x20;al., 2010</xref>; <xref ref-type="bibr" rid="B1">Adams et&#x20;al., 2013</xref>). The property closure produced in this work represents a significant advance from the closures produced in prior literature in terms of both computational cost and accuracy.</p>
</sec>
<sec id="s2">
<title>2 Background</title>
<sec id="s2-1">
<title>2.1 Microstructure Quantification</title>
<p>Discretized representations have been used extensively in the mathematical representation of the material microstructures (<xref ref-type="bibr" rid="B1">Adams et&#x20;al., 2013</xref>). In these, the microstructure is most conveniently represented as an array, <inline-formula id="inf1">
<mml:math id="m1">
<mml:mrow>
<mml:msubsup>
<mml:mi>m</mml:mi>
<mml:mi>s</mml:mi>
<mml:mi>h</mml:mi>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula>, whose values reflect the volume fraction of the material local state, <inline-formula id="inf2">
<mml:math id="m2">
<mml:mi>h</mml:mi>
</mml:math>
</inline-formula>, in the spatial bin, <inline-formula id="inf3">
<mml:math id="m3">
<mml:mi>s</mml:mi>
</mml:math>
</inline-formula>, in the RVE. In this formalism, the local states are used to index the salient local material attributes (e.g., thermodynamic phase identifiers) and the spatial bins are produced through a uniform tessellation of the RVE (equivalent to pixels in 2-D and voxels in 3-D). Implicit in this representation is the assumption that there exist a finite number of distinct material local states, <inline-formula id="inf4">
<mml:math id="m4">
<mml:mrow>
<mml:mi>h</mml:mi>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>1,2</mml:mn>
<mml:mo>,</mml:mo>
<mml:mo>&#x2026;</mml:mo>
<mml:mo>,</mml:mo>
<mml:mi>H</mml:mi>
</mml:mrow>
</mml:math>
</inline-formula>, which are allowed to occupy each spatial bin in the RVE, <inline-formula id="inf5">
<mml:math id="m5">
<mml:mrow>
<mml:mi>s</mml:mi>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>1,2</mml:mn>
<mml:mo>,</mml:mo>
<mml:mo>&#x2026;</mml:mo>
<mml:mo>,</mml:mo>
<mml:mi>S</mml:mi>
</mml:mrow>
</mml:math>
</inline-formula>, to define the microstructure of interest. In this study, we will employ a 3-D vector index <inline-formula id="inf6">
<mml:math id="m6">
<mml:mrow>
<mml:mi mathvariant="bold-italic">s</mml:mi>
<mml:mo>&#x3d;</mml:mo>
<mml:mrow>
<mml:mo>(</mml:mo>
<mml:mrow>
<mml:msub>
<mml:mi>s</mml:mi>
<mml:mn>1</mml:mn>
</mml:msub>
<mml:mo>,</mml:mo>
<mml:msub>
<mml:mi>s</mml:mi>
<mml:mn>2</mml:mn>
</mml:msub>
<mml:mo>,</mml:mo>
<mml:msub>
<mml:mi>s</mml:mi>
<mml:mn>3</mml:mn>
</mml:msub>
</mml:mrow>
<mml:mo>)</mml:mo>
</mml:mrow>
</mml:mrow>
</mml:math>
</inline-formula> for indexing the spatial bins in a 3-D&#x20;RVE.</p>
<p>Of primary interest to this paper are the 2-point spatial correlations, which are captured in a discretized representation as an array denoted by <inline-formula id="inf7">
<mml:math id="m7">
<mml:mrow>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mrow>
<mml:mi>h</mml:mi>
<mml:msup>
<mml:mi>h</mml:mi>
<mml:mo>&#x27;</mml:mo>
</mml:msup>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula>. The elements of this array reflect the probability of finding local states <inline-formula id="inf8">
<mml:math id="m8">
<mml:mi>h</mml:mi>
</mml:math>
</inline-formula> and <inline-formula id="inf9">
<mml:math id="m9">
<mml:mi>h</mml:mi>
</mml:math>
</inline-formula>&#x2032; in the RVE separated by a discretized vector indexed by an integer array <inline-formula id="inf10">
<mml:math id="m10">
<mml:mi mathvariant="bold-italic">r</mml:mi>
</mml:math>
</inline-formula> (very similar to <inline-formula id="inf11">
<mml:math id="m11">
<mml:mi mathvariant="bold-italic">s</mml:mi>
</mml:math>
</inline-formula>). Mathematically, 2-point spatial correlations are defined as (<xref ref-type="bibr" rid="B29">Kalidindi, 2015</xref>)<disp-formula id="e1">
<mml:math id="m12">
<mml:mrow>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mrow>
<mml:mi>h</mml:mi>
<mml:msup>
<mml:mi>h</mml:mi>
<mml:mo>&#x27;</mml:mo>
</mml:msup>
</mml:mrow>
</mml:msubsup>
<mml:mo>&#x3d;</mml:mo>
<mml:mfrac>
<mml:mn>1</mml:mn>
<mml:mi>S</mml:mi>
</mml:mfrac>
<mml:munder>
<mml:mstyle displaystyle="true">
<mml:mo>&#x2211;</mml:mo>
</mml:mstyle>
<mml:mi>s</mml:mi>
</mml:munder>
<mml:msubsup>
<mml:mi>m</mml:mi>
<mml:mi mathvariant="bold-italic">s</mml:mi>
<mml:mi>h</mml:mi>
</mml:msubsup>
<mml:msubsup>
<mml:mi>m</mml:mi>
<mml:mrow>
<mml:mi mathvariant="bold-italic">s</mml:mi>
<mml:mo>&#x2b;</mml:mo>
<mml:mi mathvariant="bold-italic">r</mml:mi>
</mml:mrow>
<mml:mrow>
<mml:msup>
<mml:mi>h</mml:mi>
<mml:mo>&#x27;</mml:mo>
</mml:msup>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:math>
<label>(1)</label>
</disp-formula>Where <inline-formula id="inf12">
<mml:math id="m13">
<mml:mi>S</mml:mi>
</mml:math>
</inline-formula> denotes the total number of spatial bins in the RVE. In practice, <inline-formula id="inf13">
<mml:math id="m14">
<mml:mrow>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mrow>
<mml:mi>h</mml:mi>
<mml:msup>
<mml:mi>h</mml:mi>
<mml:mo>&#x27;</mml:mo>
</mml:msup>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula> are most efficiently computed by taking advantage of the fast Fourier transform (FFT) algorithm (<xref ref-type="bibr" rid="B45">Niezgoda et&#x20;al., 2008</xref>; <xref ref-type="bibr" rid="B10">Cecen et&#x20;al., 2016</xref>).</p>
</sec>
<sec id="s2-2">
<title>2.2 Microstructure Hulls and Property Closures</title>
<p>We will restrict our attention in this work to periodic eigen microstructures. Eigen microstructures are defined as a special class of microstructures where each spatial bin is fully occupied by only one material local state. In other words, <inline-formula id="inf14">
<mml:math id="m15">
<mml:mrow>
<mml:msubsup>
<mml:mi>m</mml:mi>
<mml:mi>s</mml:mi>
<mml:mi>h</mml:mi>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula> are allowed to take only the values of either zero or one. Most experimentally observed microstructures are commonly depicted as eigen microstructures, with the spatial bin size limited by the resolution limits of the characterization machine (e.g., microscope). Moreover, most structural composites exhibit thermodynamic phase regions separated by sharp boundaries. Therefore, in practice, the discretization error arising from the use of eigen microstructure representations is largely restricted to the voxels next to the phase boundaries; this error can be controlled through the selection of a sufficiently small spatial bin size. The assumption of periodicity makes the microstructure representations consistent with the typically imposed boundary conditions in the finite element modeling of the RVEs for the estimation of their effective (bulk) mechanical properties (e.g., elastic stiffness, yield strength) (<xref ref-type="bibr" rid="B11">Cecen et&#x20;al., 2014</xref>; Brough et&#x20;al., 2017; <xref ref-type="bibr" rid="B40">Latypov et&#x20;al., 2019</xref>).</p>
<p>Prior work in the development of the MSD framework (<xref ref-type="bibr" rid="B45">Niezgoda et&#x20;al., 2008</xref>) has demonstrated that the complete space of 2-point spatial correlations (i.e.,&#x20;the set of all theoretically possible 2-point spatial correlations) can be depicted as a convex (and compact) hull. Generally referred as a microstructure hull, this construct delineates the complete space of inputs (i.e.,&#x20;design space) that needs to be considered in microstructure design. It should be recognized that the space of the 2-point spatial correlations is significantly smaller than the space of all microstructures, since microstructures related to each other by translations and/or inversions have the exact same set of 2-point spatial correlations (implied from <xref ref-type="disp-formula" rid="e1">Eq. 1</xref>). This is indeed one of the main advantages of using spatial correlations to represent the microstructure in design efforts; the microstructures that have been filtered out exhibit the exact same effective mechanical properties as the ones retained in the 2-point spatial correlations hull. Therefore, they effectively remove many of the redundancies in the design space. Although higher-order spatial correlations (i.e.,&#x20;3-point spatial correlations and higher) are known to influence the effective properties, they are expected to have (currently unknown) non-linear relationships with the 2-point spatial correlations, at least for the class of eigen microstructures considered in this work. This can be inferred from the fact that it is possible to reconstruct exactly the eigen microstructures from their 2-point spatial correlations (<xref ref-type="bibr" rid="B19">Fullwood et&#x20;al., 2008b</xref>). One of the important practical consequences of the concepts presented above is that one can construct a new set of valid 2-point spatial correlations as a convex combination of the 2-point spatial correlations of known microstructures. This realization offers an attractive avenue for exploring efficiently the space of microstructures without having to instantiate them directly. In other words, we can explore the space of <inline-formula id="inf15">
<mml:math id="m16">
<mml:mrow>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mrow>
<mml:mi>h</mml:mi>
<mml:msup>
<mml:mi>h</mml:mi>
<mml:mo>&#x27;</mml:mo>
</mml:msup>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula> much more easily than the space of <inline-formula id="inf16">
<mml:math id="m17">
<mml:mrow>
<mml:msubsup>
<mml:mi>m</mml:mi>
<mml:mi mathvariant="bold-italic">s</mml:mi>
<mml:mi>h</mml:mi>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula>. This is because it is not possible to span the space of eigen microstructures simply as convex combinations of previously known eigen microstructures.</p>
<p>Another advance from the MSD framework related to the present work is the concept of a property closure (<xref ref-type="bibr" rid="B49">Proust and Kalidindi, 2006</xref>; <xref ref-type="bibr" rid="B59">Wu et&#x20;al., 2007</xref>; <xref ref-type="bibr" rid="B17">Fullwood et&#x20;al., 2007</xref>; <xref ref-type="bibr" rid="B35">Knezevic et&#x20;al., 2008</xref>; <xref ref-type="bibr" rid="B18">Fullwood et&#x20;al., 2010</xref>; <xref ref-type="bibr" rid="B1">Adams et&#x20;al., 2013</xref>). The property closure delineates the complete set of effective (bulk) property combinations in a selected material system, which are theoretically realizable through the modulation of its microstructure. Property closures are extremely valuable in engineering design because they represent the complete set of property combinations that can be leveraged for the optimization of the part performance. This is particularly important for heterogeneous design where the microstructures are intentionally varied throughout the part to optimize the overall part performance. Prior work in the MSD framework focused on computationally efficient algorithms for mapping the microstructure hulls into property closures. As already mentioned, the case studies reported to date in the MSD framework have relied on Green&#x2019;s function-based analytical models for microstructure-property relationships, which were themselves restricted to relatively simple material physics (i.e.,&#x20;constitutive models) and low to moderate contrast composites (<xref ref-type="bibr" rid="B28">Kalidindi et&#x20;al., 2006</xref>; <xref ref-type="bibr" rid="B49">Proust and Kalidindi, 2006</xref>; <xref ref-type="bibr" rid="B1">Adams et&#x20;al., 2013</xref>).</p>
</sec>
<sec id="s2-3">
<title>2.3 Convolutional Neural Networks</title>
<p>Neural networks (<xref ref-type="bibr" rid="B53">Schmidhuber, 2015</xref>) have shown to be powerful tools for learning highly complex non-linear mappings between selected inputs and the targets (i.e.,&#x20;outputs) in a wide range of application domains. Indeed, under certain conditions, neural networks can be shown to be universal function approximators (<xref ref-type="bibr" rid="B12">Cybenko, 1989</xref>; <xref ref-type="bibr" rid="B48">Pinkus, 1999</xref>). Convolutional neural networks (CNNs) are a special class of neural networks that perform exceptionally well for problems involving spatial fields as inputs. CNNs have been successfully deployed in a variety of image analyses and machine vision applications (<xref ref-type="bibr" rid="B42">Lecun et&#x20;al., 1998</xref>; <xref ref-type="bibr" rid="B26">He et&#x20;al., 2016</xref>; <xref ref-type="bibr" rid="B37">Krizhevsky et&#x20;al., 2017</xref>). Since microstructures are spatial fields, CNNs are ideally suited to explore microstructure-property relationships (<xref ref-type="bibr" rid="B61">Yang et&#x20;al., 2018</xref>; <xref ref-type="bibr" rid="B50">Rao and Liu, 2020</xref>; <xref ref-type="bibr" rid="B13">Eidel, 2021</xref>). The central advantage of CNNs is that they circumvent the need for explicit feature engineering of the complex input spatial fields. In other words, the feature engineering occurs implicitly in the CNN during the model training process.</p>
<p>The primary components of a typical CNN are the convolutional layers, the pooling layers, and the fully connected layers, all of which are used to transform systematically the input into the desired target. <xref ref-type="fig" rid="F1">Figure&#x20;1</xref> depicts schematically a typical CNN architecture, where each block represents the transformed input (i.e.,&#x20;feature map) and the mathematical operations between the blocks are performed using one of the types of layers described above. The number of transformation layers and their characteristics (e.g., number of channels in each layer, type of non-linear activation employed, kernel size) are considered as hyperparameters of the CNN architecture, and are generally optimized for a specific application through multiple trials. As the size of the network is increased, the model accuracy and the computational cost of the training generally increases. However, increases in network size are often accompanied by increases in the number of learned (i.e.,&#x20;model-fit) parameters. Consequently, larger networks are prone to be model over-fits, especially when using a limited training dataset. Model over-fit is generally assessed through some form of cross-validation (<xref ref-type="bibr" rid="B4">Bishop, 2006</xref>; <xref ref-type="bibr" rid="B25">Hastie et&#x20;al., 2009</xref>). Therefore, one aims to build a <italic>robust</italic> CNN model that provides high model accuracy, while avoiding over-fit.</p>
<fig id="F1" position="float">
<label>FIGURE 1</label>
<caption>
<p>Schematic of a typical CNN architecture consisting of convolutional layers, pooling layers, and fully connected layers.</p>
</caption>
<graphic xlink:href="fmats-09-851085-g001.tif"/>
</fig>
<p>The reader is referred to various excellent texts (e.g., <xref ref-type="bibr" rid="B41">LeCun et&#x20;al., 2015</xref>; <xref ref-type="bibr" rid="B14">Emmert-Streib et&#x20;al., 2020</xref>; <xref ref-type="bibr" rid="B63">Zhang et&#x20;al., 2021</xref>) for an introduction to the basics of machine learning. Here, we briefly present the primary components of the CNNs discussed in this work. A convolutional layer applies a linear transformation (performed as a convolution of a learned kernel on the input) followed by a non-linear activation applied pointwise on the feature map. The PReLU activation function defined below has been used in this work:<disp-formula id="e2">
<mml:math id="m18">
<mml:mrow>
<mml:mtext>PReLU</mml:mtext>
<mml:mrow>
<mml:mo>(</mml:mo>
<mml:mtext>x</mml:mtext>
<mml:mo>)</mml:mo>
</mml:mrow>
<mml:mo>&#x3d;</mml:mo>
<mml:mrow>
<mml:mo>{</mml:mo>
<mml:mrow>
<mml:mtable>
<mml:mtr>
<mml:mtd>
<mml:mrow>
<mml:mi>x</mml:mi>
<mml:mo>,</mml:mo>
<mml:mo>&#xa0;</mml:mo>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>i</mml:mi>
<mml:mi>f</mml:mi>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>x</mml:mi>
<mml:mo>&#x2265;</mml:mo>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:mtd>
</mml:mtr>
<mml:mtr>
<mml:mtd>
<mml:mrow>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>a</mml:mi>
<mml:mi>x</mml:mi>
<mml:mo>,</mml:mo>
<mml:mo>&#xa0;</mml:mo>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>o</mml:mi>
<mml:mi>t</mml:mi>
<mml:mi>h</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>w</mml:mi>
<mml:mi>i</mml:mi>
<mml:mi>s</mml:mi>
<mml:mi>e</mml:mi>
</mml:mrow>
</mml:mtd>
</mml:mtr>
</mml:mtable>
</mml:mrow>
</mml:mrow>
</mml:mrow>
</mml:math>
<label>(2)</label>
</disp-formula>
</p>
<p>A stride (<xref ref-type="bibr" rid="B23">Goodfellow et&#x20;al., 2016</xref>; <xref ref-type="bibr" rid="B63">Zhang et&#x20;al., 2021</xref>) can be implemented in the convolution operation to effectively coarsen the input, which helps reduce the size of the transformed feature map (this is needed as most CNNs start with high-dimensional spatial maps as input and produce low dimensional targets; see <xref ref-type="fig" rid="F1">Figure&#x20;1</xref>. Pooling (<xref ref-type="bibr" rid="B23">Goodfellow et&#x20;al., 2016</xref>; <xref ref-type="bibr" rid="B63">Zhang et&#x20;al., 2021</xref>) is another dimensionality reduction technique that is used extensively in CNN architectures. Specifically, this work employs global average pooling where a feature map produced in an output channel is simply replaced by its average. Most CNN architectures implement fully connected layers in the last few transformation layers. Unlike the convolution layers, fully connected layers treat each input feature independently in the linear transformation. Consequently, fully connected layers dramatically increase the expressivity of the CNN models along with a concomitant increase in the number of model-fit parameters. Consequently, the fully connected layers also make the CNN models prone to over-fit.</p>
<p>CNNs have been successfully employed to model the microstructure-property relationships in heterogeneous (composite) material systems (<xref ref-type="bibr" rid="B61">Yang et&#x20;al., 2018</xref>; <xref ref-type="bibr" rid="B50">Rao and Liu, 2020</xref>; <xref ref-type="bibr" rid="B13">Eidel, 2021</xref>). However, the CNN architectures designed for predicting the effective property of a microstructure have not differed significantly from the CNNs designed for machine vision problems (<xref ref-type="bibr" rid="B42">Lecun et&#x20;al., 1998</xref>). For example, <xref ref-type="bibr" rid="B61">Yang et&#x20;al. (2018)</xref> employed a CNN to predict the <inline-formula id="inf17">
<mml:math id="m19">
<mml:mrow>
<mml:msub>
<mml:mi>C</mml:mi>
<mml:mrow>
<mml:mn>1111</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula> component of the effective elastic stiffness tensor of a composite using a voxelated representation of its RVE as the input. Their CNN model contained approximately 3.2 million trainable parameters, the majority of which reside in the fully connected layers. More recently, <xref ref-type="bibr" rid="B13">Eidel (2021)</xref> improved the CNN model from <xref ref-type="bibr" rid="B61">Yang et&#x20;al. (2018)</xref> by significantly reducing the number of neurons in the fully connected layers, which correspondingly reduces the number of trainable parameters. However, the CNN architecture still has approximately 2.9 million trainable parameters, of which approximately 1.8 million parameters are in the fully connected layers. Because the models are trained using relatively small datasets, typically containing only about <inline-formula id="inf18">
<mml:math id="m20">
<mml:mrow>
<mml:msup>
<mml:mrow>
<mml:mn>10</mml:mn>
</mml:mrow>
<mml:mn>4</mml:mn>
</mml:msup>
</mml:mrow>
</mml:math>
</inline-formula> (<xref ref-type="bibr" rid="B61">Yang et&#x20;al., 2018</xref>; <xref ref-type="bibr" rid="B13">Eidel, 2021</xref>) training datapoints, the chances of the CNN model being an over-fit are significant. As already discussed, the CNN architectures used in the prior microstructure-property modeling efforts exhibit the following additional limitations: 1) the use of discrete microstructures as input makes it difficult to employ them in exploring inverse microstructure-design solutions (because of the difficulty in delineating the unimaginably large input domain), and 2) they do not automatically reflect the desired translation invariance.</p>
</sec>
</sec>
<sec id="s3">
<title>3 New Protocol for Building Property Closures</title>
<p>This work proposes a new protocol for constructing property closures that leverages the prior advances made in both the MSD and MKS frameworks. More specifically, the proposed protocol combines the concepts of 2-point spatial correlations and their hulls developed in the MSD framework (<xref ref-type="bibr" rid="B1">Adams et&#x20;al., 2013</xref>) with a new CNN architecture that avoids the use of fully connected layers. The proposed protocol involves two main steps: 1) building a robust surrogate model that captures the 2-point spatial correlations-property linkage of interest using the new CNN architecture, which employs a much lower number of model fit-parameters compared to current benchmarks (<xref ref-type="bibr" rid="B61">Yang et&#x20;al., 2018</xref>; <xref ref-type="bibr" rid="B13">Eidel, 2021</xref>), and 2) constructing the property closure by systematically exploring the 2-point spatial correlations hull with the new CNN model. This new protocol is developed and demonstrated in this paper for constructing the property closure for selected components of the effective elastic stiffness tensor in a high-contrast composite material system.</p>
<sec id="s3-1">
<title>3.1 Convolutional Neural Network Model for Microstructure-Property Linkages</title>
<p>The first step of the proposed protocol for building property closures is to establish a robust surrogate model that takes 2-point spatial correlations as the input and predicts the effective properties of interest. The many benefits that could come from the use of 2-point spatial correlations as the input instead of the discrete microstructure have already been discussed earlier. It is emphasized here that the features identified by the 2-point spatial correlations are expected to serve as universal features for all effective anisotropic material properties of interest (<xref ref-type="bibr" rid="B20">Garmestani et&#x20;al., 1998</xref>; <xref ref-type="bibr" rid="B11">Cecen et&#x20;al., 2014</xref>; <xref ref-type="bibr" rid="B24">Gupta et&#x20;al., 2015</xref>; <xref ref-type="bibr" rid="B29">Kalidindi, 2015</xref>; <xref ref-type="bibr" rid="B47">Paulson et&#x20;al., 2017</xref>; <xref ref-type="bibr" rid="B60">Yabansu et&#x20;al., 2020</xref>; <xref ref-type="bibr" rid="B21">Generale and Kalidindi, 2021</xref>). Therefore, it should be possible to create microstructure-property surrogates capable of concurrently predicting multiple effective anisotropic material properties. It is further emphasized that the relationships of interest have to be necessarily formulated in the direction of microstructure &#x2192; property (and not in the inverse direction) as these are expected to be many-to-one relationships. In other words, microstructures exhibiting different 2-point spatial correlations can produce the exact same effective property, because the effective property reflects a suitably averaged bulk response of the material<xref ref-type="fn" rid="fn3">
<sup>3</sup>
</xref>.</p>
<p>A few example RVEs and their 2-point spatial correlation maps are presented in <xref ref-type="fig" rid="F2">Figure&#x20;2</xref>. It should be noted that 2-point spatial correlation maps are continuous spatial fields with a natural origin corresponding to the zero vector (i.e.,&#x20;<inline-formula id="inf19">
<mml:math id="m21">
<mml:mrow>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> in <xref ref-type="disp-formula" rid="e1">Eq. 1</xref>). Note that the 2-point spatial correlation maps exhibit a sharp peak at <inline-formula id="inf20">
<mml:math id="m22">
<mml:mrow>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> and generally decrease with increasing <inline-formula id="inf21">
<mml:math id="m23">
<mml:mrow>
<mml:mrow>
<mml:mo>&#x7c;</mml:mo>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mo>&#x7c;</mml:mo>
</mml:mrow>
</mml:mrow>
</mml:math>
</inline-formula>. More specifically, the peak value is equal to the phase volume fraction, while the asymptotic value at large <inline-formula id="inf22">
<mml:math id="m24">
<mml:mrow>
<mml:mrow>
<mml:mo>&#x7c;</mml:mo>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mo>&#x7c;</mml:mo>
</mml:mrow>
</mml:mrow>
</mml:math>
</inline-formula> is equal to the square of the phase volume fraction. Additionally, the 2-point spatial correlations capture a number of other important details of the microstructure, including size and shape distributions (<xref ref-type="bibr" rid="B18">Fullwood et&#x20;al., 2010</xref>; <xref ref-type="bibr" rid="B29">Kalidindi, 2015</xref>). Indeed, it is seen from <xref ref-type="fig" rid="F2">Figure&#x20;2</xref> that the shape of the central peak in the 2-point spatial correlations mimics the average shape of the features in the microstructure. The presence of the natural origin at <inline-formula id="inf23">
<mml:math id="m25">
<mml:mrow>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> imparts the desired translational invariance of the effective properties described earlier. The 2-point spatial correlations maps used as inputs to the CNNs developed in this work include vectors with x, y, and z components in the integer range <inline-formula id="inf24">
<mml:math id="m26">
<mml:mrow>
<mml:mrow>
<mml:mo>{</mml:mo>
<mml:mrow>
<mml:mo>&#x2212;</mml:mo>
<mml:mn>15</mml:mn>
<mml:mo>,</mml:mo>
<mml:mo>&#x2026;</mml:mo>
<mml:mo>,</mml:mo>
<mml:mo>&#xa0;</mml:mo>
<mml:mn>0</mml:mn>
<mml:mo>,</mml:mo>
<mml:mo>&#xa0;</mml:mo>
<mml:mo>&#x2026;</mml:mo>
<mml:mo>,</mml:mo>
<mml:mo>&#xa0;</mml:mo>
<mml:mn>15</mml:mn>
</mml:mrow>
<mml:mo>}</mml:mo>
</mml:mrow>
</mml:mrow>
</mml:math>
</inline-formula>. Although the vectors defining spatial correlations have physical units of length, they are used here with arbitrary units (these default to voxels of unit size in this work), as is typically done in FEM of composite RVEs. This is because the material constitutive law (i.e.,&#x20;Hooke&#x2019;s law for elasticity) employed in these models does not contain any characteristic length scales. Consequently, the values of the effective stiffness of the RVE are completely independent of the physical length of each voxel in the RVE. However, one must pay attention to the voxelization itself in creating the RVEs. The use of coarser voxels will lead to inaccurate representation of the smallest features (i.e.,&#x20;phase regions) in the RVE, while the use of finer voxels will increase the computational cost. Prior work (<xref ref-type="bibr" rid="B40">Latypov et&#x20;al., 2019</xref>; <xref ref-type="bibr" rid="B44">Marshall and Kalidindi, 2021</xref>) has utilized successfully RVEs of resolution <inline-formula id="inf25">
<mml:math id="m27">
<mml:mrow>
<mml:mn>27</mml:mn>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>X</mml:mi>
<mml:mo>&#xa0;</mml:mo>
<mml:mn>27</mml:mn>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>X</mml:mi>
<mml:mo>&#xa0;</mml:mo>
<mml:mn>27</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> in modelling the homogenized plastic response of composite microstructures. The resolution of the RVEs in this work was increased to <inline-formula id="inf26">
<mml:math id="m28">
<mml:mrow>
<mml:mn>31</mml:mn>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>X</mml:mi>
<mml:mo>&#xa0;</mml:mo>
<mml:mn>31</mml:mn>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>X</mml:mi>
<mml:mo>&#xa0;</mml:mo>
<mml:mn>31</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> to allow for a slightly improved representation of the RVEs used to capture the salient microstructure-property linkages for the present application. Further increase in the RVE resolutions was not possible for the present study because of the high computational cost involved. As a result of the considerations described above, the 2-point spatial correlation map used as input to the CNN was standardized as a 3-D array of size <inline-formula id="inf27">
<mml:math id="m29">
<mml:mrow>
<mml:mn>31</mml:mn>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>X</mml:mi>
<mml:mo>&#xa0;</mml:mo>
<mml:mn>31</mml:mn>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>X</mml:mi>
<mml:mo>&#xa0;</mml:mo>
<mml:mn>31</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> (with <inline-formula id="inf28">
<mml:math id="m30">
<mml:mrow>
<mml:mi mathvariant="bold-italic">r</mml:mi>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>0</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> corresponding to the element (16, 16, 16) of this array). Note that the elements of this array take continuous values only in the range&#x20;<inline-formula id="inf29">
<mml:math id="m31">
<mml:mrow>
<mml:mrow>
<mml:mo>[</mml:mo>
<mml:mrow>
<mml:mn>0,1</mml:mn>
</mml:mrow>
<mml:mo>]</mml:mo>
</mml:mrow>
</mml:mrow>
</mml:math>
</inline-formula>.</p>
<fig id="F2" position="float">
<label>FIGURE 2</label>
<caption>
<p>Example 3-D RVEs with the corresponding Y-Z mid-sections of their 2-point spatial correlations. It is seen that the 2-point autocorrelations capture various salient measures of the microstructures (for example, the center peak in these plots reflects the phase volume fraction and the shape of the central peak region reflects the average shape of the phase regions).</p>
</caption>
<graphic xlink:href="fmats-09-851085-g002.tif"/>
</fig>
<p>Prior work (<xref ref-type="bibr" rid="B20">Garmestani et&#x20;al., 1998</xref>; <xref ref-type="bibr" rid="B11">Cecen et&#x20;al., 2014</xref>; <xref ref-type="bibr" rid="B24">Gupta et&#x20;al., 2015</xref>; <xref ref-type="bibr" rid="B29">Kalidindi, 2015</xref>; <xref ref-type="bibr" rid="B47">Paulson et&#x20;al., 2017</xref>) has also shown that the 2-point spatial correlations can serve as universal features for correlating the microstructure to its many different effective (bulk) properties. The theoretical justification for this claim is most clearly seen in the statistical continuum theories formulated by <xref ref-type="bibr" rid="B38">Kr&#xf6;ner (1971)</xref>. Consequently, the use of 2-point spatial correlations as input can offer attractive avenues for creating high-fidelity multi-output CNN models, where each output corresponds to a different effective property of interest. In other words, one can aim to build CNN architectures that learn the common salient microstructure features that are capable of making sufficiently accurate predictions for the different effective properties of interest. Such multi-output CNNs would implicitly account for cross-correlations between the different effective properties of the RVE, making the predictions more reliable and robust. In this work, we will specifically explore multi-output CNNs for the predictions of <inline-formula id="inf30">
<mml:math id="m32">
<mml:mrow>
<mml:msub>
<mml:mi>C</mml:mi>
<mml:mrow>
<mml:mn>1111</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo>&#xa0;</mml:mo>
<mml:mtext>and</mml:mtext>
<mml:mo>&#xa0;</mml:mo>
<mml:msub>
<mml:mi>C</mml:mi>
<mml:mrow>
<mml:mn>1212</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula> components of the effective elastic stiffness tensor for high contrast composites.</p>
<p>The use of 2-point spatial correlations, instead of the voxelated microstructures, as input to the CNN essentially constitutes feature engineering. It should be recognized that most applications of CNNs do not apply any feature engineering steps. In fact, CNNs are generally touted as model building approaches that do not require feature-engineering. However, for our application, the established physics (i.e.,&#x20;statistical continuum mechanics theories (<xref ref-type="bibr" rid="B38">Kr&#xf6;ner, 1971</xref>; <xref ref-type="bibr" rid="B57">Torquato, 2002</xref>)) has already proven that the 2-point spatial correlations can serve as versatile microstructural features with a number of&#x20;desired characteristics described earlier. CNN models are ill-equipped to learn the 2-point spatial correlations from the&#x20;discrete microstructures by themselves, because the auto-correlations and cross-correlations are not easily approximated by the various transformation layers used in the CNNs. Thus, it is likely to be much more beneficial to first compute the 2-point spatial correlations as a feature engineering step, and subsequently use them as inputs into a CNN model for predicting the effective properties. One of the main benefits anticipated would be a dramatic reduction in the number of model fit parameters. A critical evaluation of this hypothesis is one of the main goals of this&#x20;work.</p>
<p>Towards the goals described above, we primarily focused on designing CNN architectures for our study that do not have any fully connected layers. After a few trials, we arrived at Model A (see <xref ref-type="table" rid="T1">Table&#x20;1</xref>) that showed better accuracy than the benchmarks with far fewer number of trainable parameters (discussed in more detail in the next section). We believe that this dramatic reduction of model complexity is attributable to the fact that we are using the 2-point spatial correlations as input to the CNN, in place of the voxelated microstructure. In order to validate this hypothesis, we created Model B (see <xref ref-type="table" rid="T1">Table&#x20;1</xref>), with the only difference from Model A coming from the use of the voxelated microstructure as the input to the CNN. It was also generally observed that the CNN models produced in this work needed far few layers compared to the benchmarks. This is because of the already feature-engineered inputs (i.e.,&#x20;2-point spatial correlations). Furthermore, the architectures explored in this work achieved the needed dimensionality reduction in the feature maps by using stride in the first convolutional layer. A final dimensionality reduction is accomplished using global average pooling in the final layer, effectively transforming the feature maps into scalar outputs (i.e.,&#x20;targets). Note that the CNN architectures explored in our work are drastically simplified compared to the current benchmarks (see <xref ref-type="table" rid="T1">Table&#x20;1</xref>).</p>
<table-wrap id="T1" position="float">
<label>TABLE 1</label>
<caption>
<p>Examples of different CNN architectures explored in this work along with the relevant benchmarks from literature. The notation a@b/c indicates a axaxa kernel, applied with a stride of c, and b channels. The default value of stride, when not mentioned, is&#x20;one.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th align="left">Model</th>
<th align="center">
<xref ref-type="bibr" rid="B61">Yang et&#x20;al. (2018)</xref>
</th>
<th align="center">
<xref ref-type="bibr" rid="B13">Eidel (2021)</xref>
</th>
<th align="center">A</th>
<th align="center">B</th>
<th align="center">C</th>
<th align="center">D</th>
</tr>
<tr>
<th align="left">Input</th>
<th align="center">51 &#xd7; 51 &#xd7; 51 Microstructure</th>
<th align="center">51 &#xd7; 51 &#xd7; 51 Microstructure</th>
<th align="center">31 &#xd7; 31 &#xd7; 31 Spatial Correlations</th>
<th align="center">31 &#xd7; 31 &#xd7; 31 Microstructure</th>
<th align="center">31 &#xd7; 31 &#xd7; 31 Spatial Correlations</th>
<th align="center">31 &#xd7; 31 &#xd7; 31 Spatial Correlations</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td align="left">L1</td>
<td align="center">3@16</td>
<td align="center">3@16</td>
<td align="center">5@8/2</td>
<td align="center">5@8/2</td>
<td align="center">5@64/2</td>
<td align="center">5@8/2</td>
</tr>
<tr>
<td align="left">L2</td>
<td align="center">MaxPool</td>
<td align="center">MaxPool</td>
<td align="center">3@16</td>
<td align="center">3@16</td>
<td align="center">3@32</td>
<td align="center">3@16</td>
</tr>
<tr>
<td align="left">L3</td>
<td align="center">3@32</td>
<td align="center">3@32</td>
<td align="center">3@32</td>
<td align="center">3@32</td>
<td align="center">3@16</td>
<td align="center">3@32</td>
</tr>
<tr>
<td align="left">L4</td>
<td align="center">MaxPool</td>
<td align="center">MaxPool</td>
<td align="center">3@64</td>
<td align="center">3@64</td>
<td align="center">3@8</td>
<td align="center">3@64</td>
</tr>
<tr>
<td align="left">L5</td>
<td align="center">3@64</td>
<td align="center">3@64</td>
<td align="center">1@2</td>
<td align="center">1@2</td>
<td align="center">1@2</td>
<td align="center">3@4</td>
</tr>
<tr>
<td align="left">L6</td>
<td align="center">MaxPool</td>
<td align="center">MaxPool</td>
<td align="center">AvgPool</td>
<td align="center">AvgPool</td>
<td align="center">AvgPool</td>
<td align="center">6@2</td>
</tr>
<tr>
<td align="left">L7</td>
<td align="center">3@128</td>
<td align="center">3@128</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
</tr>
<tr>
<td align="left">L8</td>
<td align="center">MaxPool</td>
<td align="center">MaxPool</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
</tr>
<tr>
<td align="left">L9</td>
<td align="center">3@256</td>
<td align="center">3@256</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
</tr>
<tr>
<td align="left">L10</td>
<td align="center">MaxPool</td>
<td align="center">MaxPool</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
</tr>
<tr>
<td align="left">L11</td>
<td align="center">FC-2048</td>
<td align="center">FC-256</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
</tr>
<tr>
<td align="left">L12</td>
<td align="center">FC-1024</td>
<td align="center">FC-128</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
<td align="center">&#x2014;</td>
</tr>
<tr>
<td align="left">Number of Outputs</td>
<td align="center">1</td>
<td align="center">27</td>
<td align="center">2</td>
<td align="center">2</td>
<td align="center">2</td>
<td align="center">2</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>The architectures of Models A and B follow the current benchmarks in that they start with a relatively small number of channels in the first layer and systematically increase the number of channels in the subsequent layers. Consequently, these conventional architectures aim to initially identify a smaller number of macroscale features and then further transform them in the subsequent layers into the salient features that strongly correlate with the output. We also explored the possible benefits of inverting this architecture, i.e.,&#x20;starting with a large number of channels in the first layer and systematically reducing the number of channels in subsequent layers. The general idea of these inverted architectures is that they allow the initial capture of a large number of potential features, and subsequently transform them to a smaller number of salient features. Another advantage is that these inverted architectures are more naturally aligned with the dimensionality reduction needed in our application - from the higher-dimensional input to the low-dimensional output. Model C (see <xref ref-type="table" rid="T1">Table&#x20;1</xref>) shows an example of this architecture. It was also observed that the global average pooling in the last layer was essential for producing high-fidelity CNN models for our application. In an effort to critically validate this concept, we created several CNN architectures that avoided the use of the average pooling layer and accomplished the necessary dimensionality reduction exclusively through the use of convolutional layers. Model D (see <xref ref-type="table" rid="T1">Table&#x20;1</xref>) shows one such architecture, which is very similar to Model A. Note also that the change from Model A architecture to Model D architecture actually increases the number of trainable parameters and allows for richer non-linear transformations (i.e.,&#x20;increases model expressivity). It will be shown in the next section that this increase in model expressivity dies not necessarily result in an improvement in model fidelity. The performance of Models A through D will be discussed extensively in the next section.</p>
<p>The dataset employed in this work to train the CNN networks consisted of 20,480&#x20;two-phase microstructures and their corresponding FEM-estimated <inline-formula id="inf31">
<mml:math id="m33">
<mml:mrow>
<mml:msub>
<mml:mi>C</mml:mi>
<mml:mrow>
<mml:mn>1111</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo>&#xa0;</mml:mo>
<mml:mtext>and</mml:mtext>
<mml:mo>&#xa0;</mml:mo>
<mml:msub>
<mml:mi>C</mml:mi>
<mml:mrow>
<mml:mn>1212</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula> components of the effective elastic stiffness tensor. The details of the generation of the microstructures and the finite element models used in the estimation of the effective elastic properties have been described in prior work (<xref ref-type="bibr" rid="B32">Kelly and Kalidindi, 2021</xref>). The microstructures exhibited volume fractions ranging from 0 to 100%. For the two-phase microstructures studied in this work, only the <inline-formula id="inf32">
<mml:math id="m34">
<mml:mrow>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi>r</mml:mi>
<mml:mrow>
<mml:mn>11</mml:mn>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula> are independent (<xref ref-type="bibr" rid="B45">Niezgoda et&#x20;al., 2008</xref>). Consequently, only these are used as input to the CNNs. The finite element models used for the estimation of the effective properties used periodic boundary conditions (<xref ref-type="bibr" rid="B39">Landi et&#x20;al., 2010</xref>). Specifically, the <inline-formula id="inf33">
<mml:math id="m35">
<mml:mrow>
<mml:msub>
<mml:mi>C</mml:mi>
<mml:mrow>
<mml:mn>1111</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula> component of the elastic stiffness tensor was estimated by imposing an average uniaxial strain of <inline-formula id="inf34">
<mml:math id="m36">
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mtext>&#x3b5;</mml:mtext>
<mml:mo>&#xaf;</mml:mo>
</mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mn>11</mml:mn>
</mml:mrow>
</mml:msub>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>0.001</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> on the RVE, computing the average stress <inline-formula id="inf35">
<mml:math id="m37">
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mi>&#x3c3;</mml:mi>
<mml:mo>&#xaf;</mml:mo>
</mml:mover>
</mml:mrow>
<mml:mrow>
<mml:mn>11</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula> from the FE simulation, and dividing it by the imposed strain (<inline-formula id="inf36">
<mml:math id="m38">
<mml:mrow>
<mml:mn>0.001</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula>). The <inline-formula id="inf37">
<mml:math id="m39">
<mml:mrow>
<mml:msub>
<mml:mi>C</mml:mi>
<mml:mrow>
<mml:mn>1212</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula> component of the effective elastic stiffness tensor was similarly evaluated by imposing an average shear strain. The elastic properties of the two isotropic phases used in this study were assigned using Poisson ratios <inline-formula id="inf38">
<mml:math id="m40">
<mml:mrow>
<mml:mrow>
<mml:mo>(</mml:mo>
<mml:mrow>
<mml:msub>
<mml:mtext>&#x3bd;</mml:mtext>
<mml:mn>1</mml:mn>
</mml:msub>
<mml:mo>,</mml:mo>
<mml:msub>
<mml:mtext>&#x3bd;</mml:mtext>
<mml:mn>2</mml:mn>
</mml:msub>
</mml:mrow>
<mml:mo>)</mml:mo>
</mml:mrow>
</mml:mrow>
</mml:math>
</inline-formula> and Young&#x2019;s Moduli (<inline-formula id="inf39">
<mml:math id="m41">
<mml:mrow>
<mml:msub>
<mml:mi>E</mml:mi>
<mml:mn>1</mml:mn>
</mml:msub>
<mml:mo>,</mml:mo>
<mml:msub>
<mml:mi>E</mml:mi>
<mml:mn>2</mml:mn>
</mml:msub>
<mml:mo>)</mml:mo>
</mml:mrow>
</mml:math>
</inline-formula>. The following specific values were used: <inline-formula id="inf40">
<mml:math id="m42">
<mml:mrow>
<mml:msub>
<mml:mtext>&#x3bd;</mml:mtext>
<mml:mn>1</mml:mn>
</mml:msub>
<mml:mo>&#x3d;</mml:mo>
<mml:msub>
<mml:mtext>&#x3bd;</mml:mtext>
<mml:mn>2</mml:mn>
</mml:msub>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>0.3</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula>, <inline-formula id="inf41">
<mml:math id="m43">
<mml:mrow>
<mml:msub>
<mml:mi>E</mml:mi>
<mml:mn>1</mml:mn>
</mml:msub>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>120</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> and <inline-formula id="inf42">
<mml:math id="m44">
<mml:mrow>
<mml:msub>
<mml:mi>E</mml:mi>
<mml:mn>2</mml:mn>
</mml:msub>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>6,000</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula> (as already noted, in the FEM simulations, these are associated with arbitrary but consistent units). These selections correspond to a high contrast ratio of 50 (defined by the ratio <inline-formula id="inf43">
<mml:math id="m45">
<mml:mrow>
<mml:mfrac>
<mml:mrow>
<mml:msub>
<mml:mi>E</mml:mi>
<mml:mn>2</mml:mn>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mi>E</mml:mi>
<mml:mn>1</mml:mn>
</mml:msub>
</mml:mrow>
</mml:mfrac>
</mml:mrow>
</mml:math>
</inline-formula>) for the composite material system studied here. Because of the linearity of the elasticity problem, the CNN model generated here can be applied to any composite material system with the same contrast value and the same values of the Poisson ratios, simply by applying a suitable scaling factor. It should be noted that although the level of contrast explored in this work is relatively high, it is still far less than the infinite contrast experienced in porous solids (for example, commonly encountered in additively manufactured components). We do believe that the microstructure design strategy proposed here is extendable to such extremely high contrast composites.</p>
</sec>
<sec id="s3-2">
<title>3.2 Property Closure Construction</title>
<p>In principle, the property closure should be constructed by mapping the complete 2-point spatial correlations space to the property space of interest. As already mentioned earlier, the protocol developed and implemented in this study aims to take full advantage of the fact that the complete space of the 2-point spatial correlations delineates a convex hull. The mapping between the 2-point spatial correlations and the effective properties of interest will be approximated by a suitable multi-output CNN model. Although the 2-point spatial correlations space is continuous and convex, it is still too large to explore using brute-force approaches. A clever strategy is therefore needed to successfully explore the complete space of 2-point spatial correlations and map it into the property space of interest. The following protocol (see <xref ref-type="fig" rid="F3">Figure&#x20;3</xref>) is designed and implemented in this work:<list list-type="simple">
<list-item>
<p>Step 1: Create a large initial set of voxelated eigen microstructures and compute their 2-point spatial correlations. Additionally, estimate their corresponding effective properties using suitable finite element models (i.e.,&#x20;applying periodic boundary conditions). Build an initial estimate of the property closure using this initial dataset.</p>
</list-item>
<list-item>
<p>Step 2: Using a suitable algorithm (such as a convex hull algorithm), identify the boundary points of the current estimate of the property closure. This work employed the Quickhull algorithm (<xref ref-type="bibr" rid="B3">Barber et&#x20;al., 1996</xref>), which efficiently identifies the boundary points of a convex hull defined by a set of points. These boundary points reflect extreme combinations of the properties of interest (within the current estimate of the property closure). The boundary points are updated after each iteration of the proposed protocol until the area enclosed by the boundary points does not change significantly. The microstructures corresponding to these boundary points are identified as seeds for the generation of new microstructures of interest in the next&#x20;step.</p>
</list-item>
<list-item>
<p>Step 3: Let <inline-formula id="inf44">
<mml:math id="m46">
<mml:mrow>
<mml:msup>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>a</mml:mi>
</mml:msup>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi>r</mml:mi>
<mml:mrow>
<mml:mn>11</mml:mn>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula> and <inline-formula id="inf45">
<mml:math id="m47">
<mml:mrow>
<mml:msup>
<mml:mo>&#xa0;</mml:mo>
<mml:mi>b</mml:mi>
</mml:msup>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi>r</mml:mi>
<mml:mrow>
<mml:mn>11</mml:mn>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula> represent the 2-point spatial correlations of two selected seeds identified in Step 2. Generate new microstructures by taking convex combinations of the 2-point spatial correlations of any selected pair of seeds from Step 2. In other words, create new microstructures as <inline-formula id="inf46">
<mml:math id="m48">
<mml:mrow>
<mml:mo>&#xa0;</mml:mo>
<mml:msup>
<mml:mo>&#xa0;</mml:mo>
<mml:mo>&#x2217;</mml:mo>
</mml:msup>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi>r</mml:mi>
<mml:mrow>
<mml:mn>11</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo>&#x3d;</mml:mo>
<mml:mo>&#xa0;</mml:mo>
<mml:msup>
<mml:mi>&#x3b1;</mml:mi>
<mml:mi>a</mml:mi>
</mml:msup>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi>r</mml:mi>
<mml:mrow>
<mml:mn>11</mml:mn>
</mml:mrow>
</mml:msubsup>
<mml:mo>&#x2b;</mml:mo>
<mml:msup>
<mml:mrow>
<mml:mrow>
<mml:mo>(</mml:mo>
<mml:mrow>
<mml:mn>1</mml:mn>
<mml:mo>&#x2212;</mml:mo>
<mml:mi>&#x3b1;</mml:mi>
</mml:mrow>
<mml:mo>)</mml:mo>
</mml:mrow>
</mml:mrow>
<mml:mi>b</mml:mi>
</mml:msup>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi>r</mml:mi>
<mml:mrow>
<mml:mn>11</mml:mn>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula> with <inline-formula id="inf47">
<mml:math id="m49">
<mml:mrow>
<mml:mn>0</mml:mn>
<mml:mo>&#x3c;</mml:mo>
<mml:mi>&#x3b1;</mml:mi>
<mml:mo>&#x3c;</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
</mml:math>
</inline-formula>. Create as many new microstructures as needed by selecting different seeds and varying the weights of the convex combination. Ideally, one may want to focus their expansion efforts on regions of the property closure that are currently lightly populated. In other words, take convex combinations of the 2-point spatial correlations that correspond to effective property points that reside in the sparse regions of the property closure. One can also use more than two seeds at a time. As long as the weights are positive and sum to one, the generated microstructure represents convex combination of the seeds. As an example, <xref ref-type="fig" rid="F4">Figure&#x20;4A</xref> depicts generation of new microstructures using three seeds, labelled as A, B, and C. All of the convex combinations are represented by points inside the triangle ABC. Microstructure D represents an example of such an interpolation. Using the CNN surrogate model, estimate the effective properties of interest for all of the new microstructures generated in this step. Add these new estimated properties to the set of points that currently approximates the property closure. Note the mapping between the microstructure space and the property space is expected to be highly non-linear, as illustrated in <xref ref-type="fig" rid="F4">Figure&#x20;4B</xref>. Since a CNN provides a smooth mapping between the input and the output (<xref ref-type="bibr" rid="B25">Hastie et&#x20;al., 2009</xref>), the triangular region ABC in the microstructure space would map to a (non-linearly) distorted but continuous triangular region with curvilinear sides in the property&#x20;space.</p>
</list-item>
<list-item>
<p>Step 4: In this step, we will focus on extrapolations (Step 3 only used interpolations) by essentially following the same process as in Step 3, while relaxing the requirement that all weights are positive. Extrapolations are usually produced by using at least one negative weight, while requiring the weights add to one. However, we will only allow acceptable new microstructures by requiring that all values of <inline-formula id="inf48">
<mml:math id="m50">
<mml:mrow>
<mml:msup>
<mml:mo>&#xa0;</mml:mo>
<mml:mo>&#x2217;</mml:mo>
</mml:msup>
<mml:msubsup>
<mml:mi>f</mml:mi>
<mml:mi>r</mml:mi>
<mml:mrow>
<mml:mn>11</mml:mn>
</mml:mrow>
</mml:msubsup>
</mml:mrow>
</mml:math>
</inline-formula> lie between zero and one (this condition is automatically satisfied in the interpolations in Step 3). Microstructure E in <xref ref-type="fig" rid="F4">Figure&#x20;4A</xref> represents an example of the generation of a new microstructure through an extrapolation. Once again, it might be prudent to focus the generation of new microstructures in this step to the sparsely populated regions in the current estimate of the property closure.</p>
</list-item>
<list-item>
<p>Step 5: Validate the new microstructures added in Step 3 and 4 as needed. In particular, we note that our confidence is much higher in the 2-point spatial correlations generated as interpolations in Step 3, compared to those generated in Step 4 as extrapolations. In this work, we only validated selected new points on the expanded boundaries of the property closure. For the validation, one would have to generate a discrete microstructure corresponding to the known 2-point spatial correlations using one of the established approaches in literature (<xref ref-type="bibr" rid="B19">Fullwood et&#x20;al., 2008b</xref>; <xref ref-type="bibr" rid="B51">Robertson and Kalidindi, 2021</xref>), and estimate its effective property using a suitable FEM simulation.</p>
</list-item>
<list-item>
<p>Step 6: Iterate Steps 2&#x2013;5 as needed, while continuously adding the validated new points collected in each iteration into the current estimate of the property closure. This might necessitate re-training of the CNN model after each iteration.</p>
</list-item>
</list>
</p>
<fig id="F3" position="float">
<label>FIGURE 3</label>
<caption>
<p>A flow chart of the proposed protocol.</p>
</caption>
<graphic xlink:href="fmats-09-851085-g003.tif"/>
</fig>
<fig id="F4" position="float">
<label>FIGURE 4</label>
<caption>
<p>
<bold>(A)</bold> Schematic illustration of the generation of new microstructures (defined in terms of 2-point spatial correlations) as interpolations or extrapolations. <bold>(B)</bold> Schematic depiction of the non-linear mapping of microstructures to the property space using a CNN model developed in this&#x20;work.</p>
</caption>
<graphic xlink:href="fmats-09-851085-g004.tif"/>
</fig>
<p>The central hypotheses behind the protocol described above is that the interpolations and extrapolations of the 2-point spatial correlations can effectively explore the complete microstructure space. Moreover, since the interpolations and extrapolations are conducted using promising seeds, the protocol naturally allows targeted exploration of promising regions of the property closure. As will be shown in the next section, the CNN model facilitates a sufficiently accurate non-linear mapping of the 2-point spatial correlations into the property&#x20;space.</p>
</sec>
</sec>
<sec sec-type="results|discussion" id="s4">
<title>4 Results and Discussion</title>
<p>The CNN architectures described in <xref ref-type="sec" rid="s3-1">Section 3.1</xref> were implemented in PyTorch (<xref ref-type="bibr" rid="B46">Paszke et&#x20;al., 2019</xref>), and the property closure construction protocol described in <xref ref-type="sec" rid="s3-2">Section 3.2</xref> was implemented in a Python code. The set of 20, 480 data points described above was split into independent train (60%), validation (15%), and test (25%) groups. Training was conducted on a single NVIDIA V100 GPU with 16GB of memory, utilizing the Adam optimizer (<xref ref-type="bibr" rid="B33">Kingma and Ba, 2017</xref>) with a cosine annealing learning rate (<xref ref-type="bibr" rid="B43">Loshchilov and Hutter, 2017</xref>) and a training batch size of 32. The Mean Absolute Error (MAE) loss function was utilized in this work for training the CNN model. The Mean Squared Error (MSE) loss function was also explored. However, it was found that the models trained utilizing MAE produced improved learning characteristics compared to the models trained with MSE for the present application. The CNN architectures were trained for multiple epochs until the MAE loss function converged to a minimum value. Based on multiple trials, the number of epochs was fixed at 480 epochs. Using a fixed number of epochs allows for a critical comparison of the performances of the different CNN architectures explored in this&#x20;work.</p>
<sec id="s4-1">
<title>4.1 Development of the Convolutional Neural Network Model for Microstructure-Stiffness Linkages</title>
<p>As previously described, the development of a robust CNN model requires multiple trials in which the hyperparameters of the CNN architecture are systematically varied to evaluate their influence on the model fidelity. The main types of architectures explored were summarized in <xref ref-type="table" rid="T1">Table&#x20;1</xref>. The accuracy of the different CNN models produced were evaluated using the Normalized MAE (NMAE) percentage defined as<disp-formula id="e3">
<mml:math id="m51">
<mml:mrow>
<mml:mi>N</mml:mi>
<mml:mi>M</mml:mi>
<mml:mi>A</mml:mi>
<mml:mi>E</mml:mi>
<mml:mo>&#x3d;</mml:mo>
<mml:mfrac>
<mml:mn>1</mml:mn>
<mml:mi>N</mml:mi>
</mml:mfrac>
<mml:munderover>
<mml:mstyle displaystyle="true">
<mml:mo>&#x2211;</mml:mo>
</mml:mstyle>
<mml:mrow>
<mml:mi>i</mml:mi>
<mml:mo>&#x3d;</mml:mo>
<mml:mn>1</mml:mn>
</mml:mrow>
<mml:mi>N</mml:mi>
</mml:munderover>
<mml:mrow>
<mml:mo>&#x7c;</mml:mo>
<mml:mrow>
<mml:mfrac>
<mml:mrow>
<mml:msub>
<mml:mi>S</mml:mi>
<mml:mi>i</mml:mi>
</mml:msub>
<mml:mo>&#x2212;</mml:mo>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mi>S</mml:mi>
<mml:mo>&#x5e;</mml:mo>
</mml:mover>
</mml:mrow>
<mml:mi>i</mml:mi>
</mml:msub>
</mml:mrow>
<mml:mrow>
<mml:msub>
<mml:mi>S</mml:mi>
<mml:mrow>
<mml:mi>a</mml:mi>
<mml:mi>v</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>a</mml:mi>
<mml:mi>g</mml:mi>
<mml:mi>e</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:mfrac>
</mml:mrow>
<mml:mo>&#x7c;</mml:mo>
</mml:mrow>
<mml:mo>&#xd7;</mml:mo>
<mml:mn>100</mml:mn>
</mml:mrow>
</mml:math>
<label>(3)</label>
</disp-formula>Where <inline-formula id="inf49">
<mml:math id="m52">
<mml:mrow>
<mml:msub>
<mml:mi>S</mml:mi>
<mml:mi>i</mml:mi>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula> represents the ground-truth value (established here using FEM) for the output, <inline-formula id="inf50">
<mml:math id="m53">
<mml:mrow>
<mml:msub>
<mml:mrow>
<mml:mover accent="true">
<mml:mi>S</mml:mi>
<mml:mo>&#x5e;</mml:mo>
</mml:mover>
</mml:mrow>
<mml:mi>i</mml:mi>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula> represents the CNN prediction, and <inline-formula id="inf51">
<mml:math id="m54">
<mml:mrow>
<mml:msub>
<mml:mi>S</mml:mi>
<mml:mrow>
<mml:mi>a</mml:mi>
<mml:mi>v</mml:mi>
<mml:mi>e</mml:mi>
<mml:mi>r</mml:mi>
<mml:mi>a</mml:mi>
<mml:mi>g</mml:mi>
<mml:mi>e</mml:mi>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula> denotes the ensemble average of the ground-truth values from a set of <inline-formula id="inf52">
<mml:math id="m55">
<mml:mi>N</mml:mi>
</mml:math>
</inline-formula> observations.</p>
<p>
<xref ref-type="table" rid="T2">Table&#x20;2</xref> summarizes the NMAE percentages for some of the best models produced in this work, along with the corresponding values from the benchmarks reported in literature (<xref ref-type="bibr" rid="B13">Eidel., 2021</xref>, <xref ref-type="bibr" rid="B61">Yang et&#x20;al., 2018</xref>). The table also summarizes the number of trainable parameters in each model as well as the sizes of the training/validation/test sizes employed in building and validating each model. As the table shows, the number of trainable parameters for each of the CNN models developed in this work is significantly smaller than those used in the current benchmarks. This is primarily because we have built our CNN models without using any fully connected layers. It is also worth noting that <xref ref-type="bibr" rid="B13">Eidel (2021)</xref> drastically reduced the number of neurons in the fully connected layers, when compared to <xref ref-type="bibr" rid="B61">Yang et&#x20;al. (2018)</xref>. However, even relatively smaller fully connected layers produce a very large number of trainable parameters. This is mainly because the feature maps produced at the end of the convolutional layers in the benchmark models are high-dimensional, and their reduction to a small number of features that are correlated to the outputs using the fully connected layers introduces a large number of trainable parameters. Although the <xref ref-type="bibr" rid="B13">Eidel (2021)</xref> model demonstrated higher accuracy than the <xref ref-type="bibr" rid="B61">Yang et&#x20;al. (2018)</xref> model, it should be recognized that the former used significantly more training data and a much smaller test data set. As a result of these important differences between them, it is not possible to conclude conclusively that the <xref ref-type="bibr" rid="B13">Eidel (2021)</xref> model performance is demonstrably better than that of the <xref ref-type="bibr" rid="B61">Yang et&#x20;al. (2018)</xref> model. However, it is clear from <xref ref-type="table" rid="T2">Table&#x20;2</xref> that the performances of Models A, C, and D (all of which used 2-point spatial correlations as the input) are significantly better than the benchmarks, both in terms of the prediction accuracy as well the number of trainable parameters.</p>
<table-wrap id="T2" position="float">
<label>TABLE 2</label>
<caption>
<p>Summary of the performance of selected CNN models produced in this work and their comparison with benchmarks form literature.</p>
</caption>
<table>
<thead valign="top">
<tr>
<th rowspan="2" align="left">Model</th>
<th rowspan="2" align="center">No. of Trainable Parameters</th>
<th rowspan="2" align="center">Dataset (train/valid/test)</th>
<th colspan="2" align="center">NMAE (%)</th>
</tr>
<tr>
<th align="center">
<inline-formula id="inf53">
<mml:math id="m56">
<mml:mrow>
<mml:msub>
<mml:mi>C</mml:mi>
<mml:mrow>
<mml:mn>1111</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula>
</th>
<th align="center">
<inline-formula id="inf54">
<mml:math id="m57">
<mml:mrow>
<mml:msub>
<mml:mi>C</mml:mi>
<mml:mrow>
<mml:mn>1212</mml:mn>
</mml:mrow>
</mml:msub>
</mml:mrow>
</mml:math>
</inline-formula>
</th>
</tr>
</thead>
<tbody valign="top">
<tr>
<td align="left">
<xref ref-type="bibr" rid="B61">Yang et&#x20;al. (2018)</xref>
</td>
<td align="center">3,272&#xa0;K</td>
<td align="center">3,819/1,881/2,850</td>
<td align="char" char=".">3.10</td>
<td align="center">&#x2014;</td>
</tr>
<tr>
<td align="left">
<xref ref-type="bibr" rid="B13">Eidel (2021)</xref>
</td>
<td align="center">2,981&#xa0;K</td>
<td align="center">7,000/2,000/1,000</td>
<td align="char" char=".">1.11</td>
<td align="char" char=".">1.15</td>
</tr>
<tr>
<td align="left">A</td>
<td align="center">73.9&#xa0;K</td>
<td align="center">12,288/3,072/5,210</td>
<td align="char" char=".">0.72</td>
<td align="char" char=".">0.66</td>
</tr>
<tr>
<td align="left">B</td>
<td align="center">73.9&#xa0;K</td>
<td align="center">12,288/3,072/5,210</td>
<td align="char" char=".">1.73</td>
<td align="char" char=".">1.66</td>
</tr>
<tr>
<td align="left">C</td>
<td align="center">80.8&#xa0;K</td>
<td align="center">12,288/3,072/5,210</td>
<td align="char" char=".">0.90</td>
<td align="char" char=".">0.80</td>
</tr>
<tr>
<td align="left">D</td>
<td align="center">82.5&#xa0;K</td>
<td align="center">12,288/3,072/5,210</td>
<td align="char" char=".">0.96</td>
<td align="char" char=".">0.86</td>
</tr>
</tbody>
</table>
</table-wrap>
<p>Comparing Models A and B, it becomes clear that changing the input to the CNN from the discrete microstructure to its 2-point spatial correlations produced a marked improvement in the model accuracy. This confirms the central hypotheses we laid out earlier. Since Model B exhibited comparable or better performance than the benchmarks (keeping in mind the larger and more diverse test sets used in this study) with a significantly smaller number of trainable parameters, it is argued that the CNN architectures without fully connected layers produce more robust models for our applications.</p>
<p>The improved performance of Model A over Model D underscores the need and benefits of using global average pooling as the final transformation layer for the microstructure-property CNNs. This is counter-intuitive, especially since Model D actually exhibits a higher model expressivity (i.e.,&#x20;it is capable of capturing more non-linear mappings). We believe the main reason for the improved performance of Model A over Model D is that the global average pooling in the last layer essentially serves as a model tree, where predictions from multiple models are averaged to produce the final prediction. Model tree strategies have been shown to improve the robustness of the&#x20;surrogate models in other applications (<xref ref-type="bibr" rid="B56">Ho, 1995</xref>; <xref ref-type="bibr" rid="B6">Breiman, 2001</xref>).</p>
<p>Overall, it is seen that Model A outperforms all the CNN models developed to date. Most impressively, it used only 73.9&#xa0;K model parameters. This is 1-2 orders of magnitude lower than the number of trainable parameters used in the current benchmarks for the same problem. As such, this model represents a significant advance in the proper use of CNNs in capturing the highly non-linear and elusive microstructure-property linkages of interest to materials design efforts. The high accuracy and robustness of the multi-output Model A can also be confirmed in the parity plots shown in <xref ref-type="fig" rid="F5">Figure&#x20;5</xref>. The fact that the architecture of Model A is able to make simultaneous accurate predictions for multiple effective properties opens new research avenues for future materials design efforts.</p>
<fig id="F5" position="float">
<label>FIGURE 5</label>
<caption>
<p>Parity plot showing the accuracy of Model A. The test points (red) are superimposed on the train points (blue). It is seen that both the train and test sets exhibit high levels of accuracy consistent with each&#x20;other.</p>
</caption>
<graphic xlink:href="fmats-09-851085-g005.tif"/>
</fig>
</sec>
<sec id="s4-2">
<title>4.2 Elastic Property Closure Using Convolutional Neural Network Model</title>
<p>The protocol established in <xref ref-type="sec" rid="s3-2">Section 3.2</xref> for building property closures was implemented here using Model A. <xref ref-type="fig" rid="F6">Figure&#x20;6A</xref> shows the initial estimate of the property closure using all 20,480 data points generated for training and validating the CNN model in <xref ref-type="sec" rid="s4-1">Section 4.1</xref>. It is clear that the property closure space is not well sampled by the initial dataset. This is to be expected because the protocols used to generate the microstructures only aimed to cover as many diverse microstructures as possible. They were not in any way informed by the effective properties associated with the microstructures. Also, generating the microstructures that produce a uniform sampling of the property space are especially difficult in our application, because the input is essentially a discretized 3-D eigen microstructure.</p>
<fig id="F6" position="float">
<label>FIGURE 6</label>
<caption>
<p>
<bold>(A)</bold> Original estimate of the property closure produced using the dataset generated to initially train the CNN surrogate model. <bold>(B)</bold> Updated property closure using the protocol presented in <xref ref-type="sec" rid="s3-2">Section 3.2</xref>. The red points represent the new points generated in the process of building the property closure.</p>
</caption>
<graphic xlink:href="fmats-09-851085-g006.tif"/>
</fig>
<p>As already described, the central advantage of the protocol described in <xref ref-type="sec" rid="s3-2">Section 3.2</xref> is that it allows us to generate new datapoints in selected regions of the property space. This is evident in <xref ref-type="fig" rid="F6">Figure&#x20;6B</xref>, where 26,825 new datapoints (shown in red) were generated using interpolations and extrapolations in the convex hull of the 2-point spatial correlations. Note that these new datapoints were targeted to lie in specific regions of the property closure. This ability to generate new microstructures corresponding to selected regions of the property space at low computational cost is unprecedented, and is only possible because the CNN model was established using 2-point spatial correlations as the input. The central consequence of the property closure shown in <xref ref-type="fig" rid="F6">Figure&#x20;6B</xref> is that it is now possible to trivially produce a large number of microstructures that correspond to any designer-specified combination of properties within the property closure.</p>
<p>It is emphasized here that the property closure presented in <xref ref-type="fig" rid="F6">Figure&#x20;6B</xref> is the first of its kind. All previously reported property closures either used grossly simplified descriptions of the microstructure (e.g., one-point statistics) or substantially degraded models (e.g., truncated expansions, primitive bounds). As such, the property closure presented in <xref ref-type="fig" rid="F6">Figure&#x20;6B</xref> represents the most accurate depiction to date of the property closure for the selected problem. Although we restricted our attention in this work to a two-phase composite with a high-contrast in the elastic properties of its constituent phases, the framework presented here is extensible to much more complicated class of composite (i.e.,&#x20;heterogeneous) microstructures and their different properties of interest (e.g., yield strength, conductivity, permeability).</p>
</sec>
</sec>
<sec id="s5">
<title>5 Conclusion</title>
<p>In this work, a new CNN architecture is proposed that takes as input the 2-point spatial correlations of a voxelated eigen microstructure and predicts its effective properties of interest. Although CNNs are generally viewed as a model building technique that bypasses explicit feature engineering, it was observed that transforming the voxelated microstructure into its 2-point spatial correlations before inputting them into the CNN model dramatically improved the model accuracy and robustness. Specifically, it was shown that it is possible to build CNN models exhibiting &#x223c;0.7% test NMAE for simultaneous predictions of two different elastic stiffness components for a high-contrast (&#x3d;50) 3-D composite microstructure. This unprecedented model accuracy and robustness was made possible by avoiding the use of fully connected layers, using a global average pooling in the final layer, and using 2-point spatial correlations as input to the CNN. It was also demonstrated that the CNN model produced in this work is capable of producing the most accurate elastic property closure available today for the selected high-contrast composite material system.</p>
</sec>
</body>
<back>
<sec id="s6">
<title>Data Availability Statement</title>
<p>The datasets presented in this study can be found in online repositories. The names of the repository/repositories and accession number(s) can be found below: <ext-link ext-link-type="uri" xlink:href="https://www.dropbox.com/sh/i9yls9d8aba2sy6/AABG-MAWABZ9gS947Phx7kgya?dl=0">https://www.dropbox.com/sh/i9yls9d8aba2sy6/AABG-MAWABZ9gS947Phx7kgya?dl&#x3d;0</ext-link>
</p>
</sec>
<sec id="s7">
<title>Author Contributions</title>
<p>AM and SK conceptualized the work. AM developed the code and generated figures and tables. AM and SK wrote and edited the manuscript.</p>
</sec>
<sec id="s8">
<title>Funding</title>
<p>The authors acknowledge funding from NSF 2027105. This work utilized the Hive computing cluster supported by NSF 1828187 and managed by PACE at Georgia Institute of Technology, United&#x20;States.</p>
</sec>
<sec sec-type="COI-statement" id="s9">
<title>Conflict of Interest</title>
<p>The authors declare that the research was conducted in the absence of any commercial or financial relationships that could be construed as a potential conflict of interest.</p>
</sec>
<sec sec-type="disclaimer" id="s10">
<title>Publisher&#x2019;s Note</title>
<p>All claims expressed in this article are solely those of the authors and do not necessarily represent those of their affiliated organizations, or those of the publisher, the editors and the reviewers. Any product that may be evaluated in this article, or claim that may be made by its manufacturer, is not guaranteed or endorsed by the publisher.</p>
</sec>
<ack>
<p>The authors thank Conlain Kelly and Andreas Robertson for their helpful comments in guiding this&#x20;work.</p>
</ack>
<fn-group>
<fn id="fn1">
<label>1</label>
<p>This term is generally used to refer to the salient details of the material internal structure which often includes statistical information on the size, shape, and placement of the different material local states (e.g., thermodynamic phases).</p>
</fn>
<fn id="fn2">
<label>2</label>
<p>This implies that if one extends the original RVE in all directions utilizing periodicity and takes a new RVE of the same size but with a different starting point, its effective property would be exactly the same as the original&#x20;RVE.</p>
</fn>
<fn id="fn3">
<label>3</label>
<p>The reader is pointed to prior work on iso-property surfaces in microstructure hulls (<xref ref-type="bibr" rid="B34">Knezevic and Kalidindi, 2007</xref>; <xref ref-type="bibr" rid="B18">Fullwood et&#x20;al., 2010</xref>) to visualize the many-to-one microstructure-property linkages.</p>
</fn>
</fn-group>
<ref-list>
<title>References</title>
<ref id="B1">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Adams</surname>
<given-names>B. L.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Fullwood</surname>
<given-names>D. T.</given-names>
</name>
</person-group> (<year>2013</year>). <source>Microstructure-sensitive Design for Performance Optimization</source>. <publisher-loc>Waltham, MA</publisher-loc>: <publisher-name>Butterworth-Heinemann</publisher-name> </citation>
</ref>
<ref id="B2">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Adams</surname>
<given-names>B. L.</given-names>
</name>
<name>
<surname>Henrie</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Henrie</surname>
<given-names>B.</given-names>
</name>
<name>
<surname>Lyon</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Garmestani</surname>
<given-names>H.</given-names>
</name>
</person-group> (<year>2001</year>). <article-title>Microstructure-sensitive Design of a Compliant Beam</article-title>. <source>J.&#x20;Mech. Phys. Sol.</source> <volume>49</volume>, <fpage>1639</fpage>&#x2013;<lpage>1663</lpage>. <pub-id pub-id-type="doi">10.1016/S0022-5096(01)00016-3</pub-id> </citation>
</ref>
<ref id="B3">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Barber</surname>
<given-names>C. B.</given-names>
</name>
<name>
<surname>Dobkin</surname>
<given-names>D. P.</given-names>
</name>
<name>
<surname>Huhdanpaa</surname>
<given-names>H.</given-names>
</name>
</person-group> (<year>1996</year>). <article-title>The Quickhull Algorithm for Convex Hulls</article-title>. <source>ACM Trans. Math. Softw.</source> <volume>22</volume>, <fpage>469</fpage>&#x2013;<lpage>483</lpage>. <pub-id pub-id-type="doi">10.1145/235815.235821</pub-id> </citation>
</ref>
<ref id="B4">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Bishop</surname>
<given-names>C. M.</given-names>
</name>
</person-group> (<year>2006</year>). <source>Pattern Recognition and Machine Learning, Information Science and Statistics</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Springer</publisher-name>. </citation>
</ref>
<ref id="B5">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Brands</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Balzani</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Scheunemann</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Schr&#xf6;der</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Richter</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Raabe</surname>
<given-names>D.</given-names>
</name>
</person-group> (<year>2016</year>). <article-title>Computational Modeling of Dual-phase Steels Based on Representative Three-Dimensional Microstructures Obtained from EBSD Data</article-title>. <source>Arch. Appl. Mech.</source> <volume>86</volume>, <fpage>575</fpage>&#x2013;<lpage>598</lpage>. <pub-id pub-id-type="doi">10.1007/s00419-015-1044-1</pub-id> </citation>
</ref>
<ref id="B6">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Breiman</surname>
<given-names>L.</given-names>
</name>
</person-group> (<year>2001</year>). <article-title>Random Forests</article-title>. <source>Mach. Learn.</source> <volume>45</volume>, <fpage>5</fpage>&#x2013;<lpage>32</lpage>. <pub-id pub-id-type="doi">10.1023/A:1010933404324</pub-id> </citation>
</ref>
<ref id="B7">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Brough</surname>
<given-names>D. B.</given-names>
</name>
<name>
<surname>Wheeler</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2017a</year>). <article-title>Materials Knowledge Systems in Python-A Data Science Framework for Accelerated Development of Hierarchical Materials</article-title>. <source>Integr. Mater. Manuf Innov.</source> <volume>6</volume>, <fpage>36</fpage>&#x2013;<lpage>53</lpage>. <pub-id pub-id-type="doi">10.1007/s40192-017-0089-0</pub-id> </citation>
</ref>
<ref id="B8">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Brough</surname>
<given-names>D. B.</given-names>
</name>
<name>
<surname>Wheeler</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Warren</surname>
<given-names>J.&#x20;A.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2017b</year>). <article-title>Microstructure-based Knowledge Systems for Capturing Process-Structure Evolution Linkages</article-title>. <source>Curr. Opin. Solid State. Mater. Sci.</source> <volume>21</volume>, <fpage>129</fpage>&#x2013;<lpage>140</lpage>. <pub-id pub-id-type="doi">10.1016/j.cossms.2016.05.002</pub-id> </citation>
</ref>
<ref id="B9">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Cecen</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Dai</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Yabansu</surname>
<given-names>Y. C.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Song</surname>
<given-names>L.</given-names>
</name>
</person-group> (<year>2018</year>). <article-title>Material Structure-Property Linkages Using Three-Dimensional Convolutional Neural Networks</article-title>. <source>Acta Materialia</source> <volume>146</volume>, <fpage>76</fpage>&#x2013;<lpage>84</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2017.11.053</pub-id> </citation>
</ref>
<ref id="B10">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Cecen</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Fast</surname>
<given-names>T.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2016</year>). <article-title>Versatile Algorithms for the Computation of 2-point Spatial Correlations in Quantifying Material Structure</article-title>. <source>Integr. Mater. Manuf Innov.</source> <volume>5</volume>, <fpage>1</fpage>&#x2013;<lpage>15</lpage>. <pub-id pub-id-type="doi">10.1186/s40192-015-0044-x</pub-id> </citation>
</ref>
<ref id="B11">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>&#xc7;e&#xe7;en</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Fast</surname>
<given-names>T.</given-names>
</name>
<name>
<surname>Kumbur</surname>
<given-names>E. C.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2014</year>). <article-title>A Data-Driven Approach to Establishing Microstructure-Property Relationships in Porous Transport Layers of Polymer Electrolyte Fuel Cells</article-title>. <source>J.&#x20;Power Sourc.</source> <volume>245</volume>, <fpage>144</fpage>&#x2013;<lpage>153</lpage>. <pub-id pub-id-type="doi">10.1016/j.jpowsour.2013.06.100</pub-id> </citation>
</ref>
<ref id="B12">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Cybenko</surname>
<given-names>G.</given-names>
</name>
</person-group> (<year>1989</year>). <article-title>Approximation by Superpositions of a Sigmoidal Function</article-title>. <source>Math. Control. Signal. Syst.</source> <volume>2</volume>, <fpage>303</fpage>&#x2013;<lpage>314</lpage>. <pub-id pub-id-type="doi">10.1007/BF02551274</pub-id> </citation>
</ref>
<ref id="B13">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Eidel</surname>
<given-names>B.</given-names>
</name>
</person-group> (<year>2021</year>). <source>Deep Convolutional Neural Networks Predict Elasticity Tensors and Their Bounds in Homogenization</source>. <comment>Preprint. arXiv:2109.03020</comment>. </citation>
</ref>
<ref id="B14">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Emmert-Streib</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Yang</surname>
<given-names>Z.</given-names>
</name>
<name>
<surname>Feng</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Tripathi</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Dehmer</surname>
<given-names>M.</given-names>
</name>
</person-group> (<year>2020</year>). <article-title>An Introductory Review of Deep Learning for Prediction Models with Big Data</article-title>. <source>Front. Artif. Intell.</source> <volume>3</volume>, <fpage>4</fpage>. <pub-id pub-id-type="doi">10.3389/frai.2020.00004</pub-id> </citation>
</ref>
<ref id="B15">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Fast</surname>
<given-names>T.</given-names>
</name>
<name>
<surname>Knezevic</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2008</year>). <article-title>Application of Microstructure Sensitive Design to Structural Components Produced from Hexagonal Polycrystalline Metals</article-title>. <source>Comput. Mater. Sci.</source> <volume>43</volume>, <fpage>374</fpage>&#x2013;<lpage>383</lpage>. <pub-id pub-id-type="doi">10.1016/j.commatsci.2007.12.002</pub-id> </citation>
</ref>
<ref id="B16">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Fullwood</surname>
<given-names>D. T.</given-names>
</name>
<name>
<surname>Adams</surname>
<given-names>B. L.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2008a</year>). <article-title>A strong Contrast Homogenization Formulation for Multi-phase Anisotropic Materials</article-title>. <source>J.&#x20;Mech. Phys. Sol.</source> <volume>56</volume>, <fpage>2287</fpage>&#x2013;<lpage>2297</lpage>. <pub-id pub-id-type="doi">10.1016/j.jmps.2008.01.003</pub-id> </citation>
</ref>
<ref id="B17">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Fullwood</surname>
<given-names>D. T.</given-names>
</name>
<name>
<surname>Adams</surname>
<given-names>B. L.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2007</year>). <article-title>Generalized Pareto Front Methods Applied to Second-Order Material Property Closures</article-title>. <source>Comput. Mater. Sci.</source> <volume>38</volume>, <fpage>788</fpage>&#x2013;<lpage>799</lpage>. <pub-id pub-id-type="doi">10.1016/j.commatsci.2006.05.016</pub-id> </citation>
</ref>
<ref id="B18">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Fullwood</surname>
<given-names>D. T.</given-names>
</name>
<name>
<surname>Niezgoda</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Adams</surname>
<given-names>B. L.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>Microstructure Sensitive Design for Performance Optimization</article-title>. <source>Prog. Mater. Sci.</source> <volume>55</volume>, <fpage>477</fpage>&#x2013;<lpage>562</lpage>. <pub-id pub-id-type="doi">10.1016/j.pmatsci.2009.08.002</pub-id> </citation>
</ref>
<ref id="B19">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Fullwood</surname>
<given-names>D. T.</given-names>
</name>
<name>
<surname>Niezgoda</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2008b</year>). <article-title>Microstructure Reconstructions from 2-point Statistics Using Phase-Recovery Algorithms</article-title>. <source>Acta Materialia</source> <volume>56</volume>, <fpage>942</fpage>&#x2013;<lpage>948</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2007.10.044</pub-id> </citation>
</ref>
<ref id="B20">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Garmestani</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Lin</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Adams</surname>
<given-names>B. L.</given-names>
</name>
</person-group> (<year>1998</year>). <article-title>Statistical Continuum Theory for Inelastic Behavior of a Two-phase Medium</article-title>. <source>Int. J.&#x20;Plasticity</source> <volume>14</volume>, <fpage>719</fpage>&#x2013;<lpage>731</lpage>. <pub-id pub-id-type="doi">10.1016/S0749-6419(98)00019-9</pub-id> </citation>
</ref>
<ref id="B21">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Generale</surname>
<given-names>A. P.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2021</year>). <article-title>Reduced-order Models for Microstructure-Sensitive Effective thermal Conductivity of Woven Ceramic Matrix Composites with Residual Porosity</article-title>. <source>Compos. Structures</source> <volume>274</volume>, <fpage>114399</fpage>. <pub-id pub-id-type="doi">10.1016/j.compstruct.2021.114399</pub-id> </citation>
</ref>
<ref id="B22">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Ghosh</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Lee</surname>
<given-names>K.</given-names>
</name>
<name>
<surname>Moorthy</surname>
<given-names>S.</given-names>
</name>
</person-group> (<year>1995</year>). <article-title>Multiple Scale Analysis of Heterogeneous Elastic Structures Using Homogenization Theory and Voronoi Cell Finite Element Method</article-title>. <source>Int. J.&#x20;Sol. Structures</source> <volume>32</volume>, <fpage>27</fpage>&#x2013;<lpage>62</lpage>. <pub-id pub-id-type="doi">10.1016/0020-7683(94)00097-G</pub-id> </citation>
</ref>
<ref id="B23">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Goodfellow</surname>
<given-names>I.</given-names>
</name>
<name>
<surname>Bengio</surname>
<given-names>Y.</given-names>
</name>
<name>
<surname>Courville</surname>
<given-names>A.</given-names>
</name>
</person-group> (<year>2016</year>). <source>Deep Learning, Adaptive Computation and Machine Learning</source>. <publisher-loc>Cambridge, Massachusetts</publisher-loc>: <publisher-name>The MIT Press</publisher-name>. </citation>
</ref>
<ref id="B24">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Gupta</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Cecen</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Goyal</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Singh</surname>
<given-names>A. K.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>Structure-property Linkages Using a Data Science Approach: Application to a Non-metallic Inclusion/steel Composite System</article-title>. <source>Acta Materialia</source> <volume>91</volume>, <fpage>239</fpage>&#x2013;<lpage>254</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2015.02.045</pub-id> </citation>
</ref>
<ref id="B25">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Hastie</surname>
<given-names>T.</given-names>
</name>
<name>
<surname>Tibshirani</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Friedman</surname>
<given-names>J.&#x20;H.</given-names>
</name>
</person-group> (<year>2009</year>). <article-title>The Elements of Statistical Learning: Data Mining, Inference, and Prediction</article-title>. in&#x201d; <source>Springer Series in Statistics</source>. <edition>2nd ed.</edition> (<publisher-loc>New York, NY</publisher-loc>: <publisher-name>Springer</publisher-name>) </citation>
</ref>
<ref id="B26">
<citation citation-type="confproc">
<person-group person-group-type="author">
<name>
<surname>He</surname>
<given-names>K.</given-names>
</name>
<name>
<surname>Zhang</surname>
<given-names>X.</given-names>
</name>
<name>
<surname>Ren</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Sun</surname>
<given-names>J.</given-names>
</name>
</person-group> (<year>2016</year>). <article-title>Deep Residual Learning for Image Recognition</article-title>. in&#x201d; <conf-name>2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Presented at the 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR)</conf-name>. <publisher-loc>USA</publisher-loc>: <publisher-name>IEEE, Las VegasNV</publisher-name>, <fpage>770</fpage>&#x2013;<lpage>778</lpage>. <pub-id pub-id-type="doi">10.1109/CVPR.2016.90</pub-id> </citation>
</ref>
<ref id="B27">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Jung</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Yoon</surname>
<given-names>J.&#x20;I.</given-names>
</name>
<name>
<surname>Park</surname>
<given-names>H. K.</given-names>
</name>
<name>
<surname>Jo</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Kim</surname>
<given-names>H. S.</given-names>
</name>
</person-group> (<year>2020</year>). <article-title>Microstructure Design Using Machine Learning Generated Low Dimensional and Continuous Design Space</article-title>. <source>Materialia</source> <volume>11</volume>, <fpage>100690</fpage>. <pub-id pub-id-type="doi">10.1016/j.mtla.2020.100690</pub-id> </citation>
</ref>
<ref id="B28">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Binci</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Fullwood</surname>
<given-names>D.</given-names>
</name>
<name>
<surname>Adams</surname>
<given-names>B. L.</given-names>
</name>
</person-group> (<year>2006</year>). <article-title>Elastic Properties Closures Using Second-Order Homogenization Theories: Case Studies in Composites of Two Isotropic Constituents</article-title>. <source>Acta Materialia</source> <volume>54</volume>, <fpage>3117</fpage>&#x2013;<lpage>3126</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2006.03.005</pub-id> </citation>
</ref>
<ref id="B29">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2015</year>). <source>Hierarchical Materials Informatics: Novel Analytics for Materials Data</source>. <publisher-loc>Amsterdam</publisher-loc>: <publisher-name>Elsevier</publisher-name>. </citation>
</ref>
<ref id="B30">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Niezgoda</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Landi</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Vachhani</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Fast</surname>
<given-names>T.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>A Novel Framework for Building Materials Knowledge Systems</article-title>. <source>Comput. Mater. Contin.</source> <volume>17</volume>, <fpage>103</fpage>. <pub-id pub-id-type="doi">10.3970/cmc.2010.017.103</pub-id> </citation>
</ref>
<ref id="B31">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Schoenfeld</surname>
<given-names>S. E.</given-names>
</name>
</person-group> (<year>2000</year>). <article-title>On the Prediction of Yield Surfaces by the crystal Plasticity Models for Fcc Polycrystals</article-title>. <source>Mater. Sci. Eng. A</source> <volume>293</volume>, <fpage>120</fpage>&#x2013;<lpage>129</lpage>. <pub-id pub-id-type="doi">10.1016/S0921-5093(00)01048-0</pub-id> </citation>
</ref>
<ref id="B32">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Kelly</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2021</year>). <article-title>Recurrent Localization Networks Applied to the Lippmann-Schwinger Equation</article-title>. <source>Comput. Mater. Sci.</source> <volume>192</volume>, <fpage>110356</fpage>. <pub-id pub-id-type="doi">10.1016/j.commatsci.2021.110356</pub-id> </citation>
</ref>
<ref id="B33">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Kingma</surname>
<given-names>D. P.</given-names>
</name>
<name>
<surname>Ba</surname>
<given-names>J.</given-names>
</name>
</person-group> (<year>2017</year>). <source>Adam: A Method for Stochastic Optimization</source>. <comment>Preprint. arXiv:1412.6980 Cs</comment>. </citation>
</ref>
<ref id="B34">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Knezevic</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2007</year>). <article-title>Fast Computation of First-Order Elastic-Plastic Closures for Polycrystalline Cubic-Orthorhombic Microstructures</article-title>. <source>Comput. Mater. Sci.</source> <volume>39</volume>, <fpage>643</fpage>&#x2013;<lpage>648</lpage>. <pub-id pub-id-type="doi">10.1016/j.commatsci.2006.08.025</pub-id> </citation>
</ref>
<ref id="B35">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Knezevic</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Mishra</surname>
<given-names>R. K.</given-names>
</name>
</person-group> (<year>2008</year>). <article-title>Delineation of First-Order Closures for Plastic Properties Requiring Explicit Consideration of Strain Hardening and Crystallographic Texture Evolution</article-title>. <source>Int. J.&#x20;Plasticity</source> <volume>24</volume>, <fpage>327</fpage>&#x2013;<lpage>342</lpage>. <pub-id pub-id-type="doi">10.1016/j.ijplas.2007.05.002</pub-id> </citation>
</ref>
<ref id="B36">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Kollmann</surname>
<given-names>H. T.</given-names>
</name>
<name>
<surname>Abueidda</surname>
<given-names>D. W.</given-names>
</name>
<name>
<surname>Koric</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Guleryuz</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Sobh</surname>
<given-names>N. A.</given-names>
</name>
</person-group> (<year>2020</year>). <article-title>Deep Learning for Topology Optimization of 2D Metamaterials</article-title>. <source>Mater. Des.</source> <volume>196</volume>, <fpage>109098</fpage>. <pub-id pub-id-type="doi">10.1016/j.matdes.2020.109098</pub-id> </citation>
</ref>
<ref id="B37">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Krizhevsky</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Sutskever</surname>
<given-names>I.</given-names>
</name>
<name>
<surname>Hinton</surname>
<given-names>G. E.</given-names>
</name>
</person-group> (<year>2017</year>). <article-title>ImageNet Classification with Deep Convolutional Neural Networks</article-title>. <source>Commun. ACM</source> <volume>60</volume>, <fpage>84</fpage>&#x2013;<lpage>90</lpage>. <pub-id pub-id-type="doi">10.1145/3065386</pub-id> </citation>
</ref>
<ref id="B38">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Kr&#xf6;ner</surname>
<given-names>E.</given-names>
</name>
</person-group> (<year>1971</year>). <source>Statistical Continuum Mechanics. Statistical Continuum Mechanics, CISM International Centre for Mechanical Sciences</source>. <publisher-loc>Vienna</publisher-loc>: <publisher-name>Springer Vienna</publisher-name>. <pub-id pub-id-type="doi">10.1007/978-3-7091-2862-6</pub-id> </citation>
</ref>
<ref id="B39">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Landi</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Niezgoda</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>Multi-scale Modeling of Elastic Response of Three-Dimensional Voxel-Based Microstructure Datasets Using Novel DFT-Based Knowledge Systems</article-title>. <source>Acta Materialia</source> <volume>58</volume>, <fpage>2716</fpage>&#x2013;<lpage>2725</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2010.01.007</pub-id> </citation>
</ref>
<ref id="B40">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Latypov</surname>
<given-names>M. I.</given-names>
</name>
<name>
<surname>Toth</surname>
<given-names>L. S.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2019</year>). <article-title>Materials Knowledge System for Nonlinear Composites</article-title>. <source>Comp. Methods Appl. Mech. Eng.</source> <volume>346</volume>, <fpage>180</fpage>&#x2013;<lpage>196</lpage>. <pub-id pub-id-type="doi">10.1016/j.cma.2018.11.034</pub-id> </citation>
</ref>
<ref id="B41">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>LeCun</surname>
<given-names>Y.</given-names>
</name>
<name>
<surname>Bengio</surname>
<given-names>Y.</given-names>
</name>
<name>
<surname>Hinton</surname>
<given-names>G.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>Deep Learning</article-title>. <source>Nature</source> <volume>521</volume>, <fpage>436</fpage>&#x2013;<lpage>444</lpage>. <pub-id pub-id-type="doi">10.1038/nature14539</pub-id> </citation>
</ref>
<ref id="B42">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Lecun</surname>
<given-names>Y.</given-names>
</name>
<name>
<surname>Bottou</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Bengio</surname>
<given-names>Y.</given-names>
</name>
<name>
<surname>Haffner</surname>
<given-names>P.</given-names>
</name>
</person-group> (<year>1998</year>). <article-title>Gradient-based Learning Applied to Document Recognition</article-title>. <source>Proc. IEEE</source> <volume>86</volume>, <fpage>2278</fpage>&#x2013;<lpage>2324</lpage>. <pub-id pub-id-type="doi">10.1109/5.726791</pub-id> </citation>
</ref>
<ref id="B43">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Loshchilov</surname>
<given-names>I.</given-names>
</name>
<name>
<surname>Hutter</surname>
<given-names>F.</given-names>
</name>
</person-group> (<year>2017</year>). <source>SGDR: Stochastic Gradient Descent with Warm Restarts</source>. <comment>Preprint. arXiv:1608.03983 Cs Math</comment>. </citation>
</ref>
<ref id="B44">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Marshall</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2021</year>). <article-title>Autonomous Development of a Machine-Learning Model for the Plastic Response of Two-phase Composites from Micromechanical Finite Element Models</article-title>. <source>JOM</source> <volume>73</volume>, <fpage>2085</fpage>&#x2013;<lpage>2095</lpage>. <pub-id pub-id-type="doi">10.1007/s11837-021-04696-w</pub-id> </citation>
</ref>
<ref id="B45">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Niezgoda</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Fullwood</surname>
<given-names>D. T.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2008</year>). <article-title>Delineation of the Space of 2-point Correlations in a Composite Material System</article-title>. <source>Acta Materialia</source> <volume>56</volume>, <fpage>5285</fpage>&#x2013;<lpage>5292</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2008.07.005</pub-id> </citation>
</ref>
<ref id="B46">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Paszke</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Gross</surname>
<given-names>S.</given-names>
</name>
<name>
<surname>Massa</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Lerer</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Bradbury</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Chanan</surname>
<given-names>G.</given-names>
</name>
<etal/>
</person-group> (<year>2019</year>). <source>PyTorch: An Imperative Style, High-Performance Deep Learning Library</source>. <comment>Preprint. arXiv:1912.01703</comment>. Editors <person-group person-group-type="editor">
<name>
<surname>Wallach</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Larochelle</surname>
<given-names>H.</given-names>
</name>
<name>
<surname>Beygelzimer</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Alch&#xe9;-Buc</surname>
<given-names>F. d&#x2019;</given-names>
</name>
<name>
<surname>Fox</surname>
<given-names>E.</given-names>
</name>
<name>
<surname>Garnett</surname>
<given-names>R.</given-names>
</name>
</person-group> (<publisher-name>Curran Associates, Inc</publisher-name>). </citation>
</ref>
<ref id="B47">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Paulson</surname>
<given-names>N. H.</given-names>
</name>
<name>
<surname>Priddy</surname>
<given-names>M. W.</given-names>
</name>
<name>
<surname>McDowell</surname>
<given-names>D. L.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2017</year>). <article-title>Reduced-order Structure-Property Linkages for Polycrystalline Microstructures Based on 2-point Statistics</article-title>. <source>Acta Materialia</source> <volume>129</volume>, <fpage>428</fpage>&#x2013;<lpage>438</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2017.03.009</pub-id> </citation>
</ref>
<ref id="B48">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Pinkus</surname>
<given-names>A.</given-names>
</name>
</person-group> (<year>1999</year>). <article-title>Approximation Theory of the MLP Model in Neural Networks</article-title>. <source>Acta Numerica</source> <volume>8</volume>, <fpage>143</fpage>&#x2013;<lpage>195</lpage>. <pub-id pub-id-type="doi">10.1017/S0962492900002919</pub-id> </citation>
</ref>
<ref id="B49">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Proust</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S.</given-names>
</name>
</person-group> (<year>2006</year>). <article-title>Procedures for Construction of Anisotropic Elastic-Plastic Property Closures for Face-Centered Cubic Polycrystals Using First-Order Bounding Relations</article-title>. <source>J.&#x20;Mech. Phys. Sol.</source> <volume>54</volume>, <fpage>1744</fpage>&#x2013;<lpage>1762</lpage>. <pub-id pub-id-type="doi">10.1016/j.jmps.2006.01.010</pub-id> </citation>
</ref>
<ref id="B50">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Rao</surname>
<given-names>C.</given-names>
</name>
<name>
<surname>Liu</surname>
<given-names>Y.</given-names>
</name>
</person-group> (<year>2020</year>). <article-title>Three-dimensional Convolutional Neural Network (3D-CNN) for Heterogeneous Material Homogenization</article-title>. <source>Comput. Mater. Sci.</source> <volume>184</volume>, <fpage>109850</fpage>. <pub-id pub-id-type="doi">10.1016/j.commatsci.2020.109850</pub-id> </citation>
</ref>
<ref id="B51">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Robertson</surname>
<given-names>A. E.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2021</year>). <article-title>Efficient Generation of Anisotropic N-Field Microstructures from 2-Point Statistics Using Multi-Output Gaussian Random Fields</article-title>. <source>SSRN J</source> <comment>In Press</comment>. <pub-id pub-id-type="doi">10.2139/ssrn.3949516</pub-id> </citation>
</ref>
<ref id="B52">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Roters</surname>
<given-names>F.</given-names>
</name>
<name>
<surname>Eisenlohr</surname>
<given-names>P.</given-names>
</name>
<name>
<surname>Hantcherli</surname>
<given-names>L.</given-names>
</name>
<name>
<surname>Tjahjanto</surname>
<given-names>D. D.</given-names>
</name>
<name>
<surname>Bieler</surname>
<given-names>T. R.</given-names>
</name>
<name>
<surname>Raabe</surname>
<given-names>D.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>Overview of Constitutive Laws, Kinematics, Homogenization and Multiscale Methods in crystal Plasticity Finite-Element Modeling: Theory, Experiments, Applications</article-title>. <source>Acta Materialia</source> <volume>58</volume>, <fpage>1152</fpage>&#x2013;<lpage>1211</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2009.10.058</pub-id> </citation>
</ref>
<ref id="B53">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Schmidhuber</surname>
<given-names>J.</given-names>
</name>
</person-group> (<year>2015</year>). <article-title>Deep Learning in Neural Networks: An Overview</article-title>. <source>Neural Networks</source> <volume>61</volume>, <fpage>85</fpage>&#x2013;<lpage>117</lpage>. <pub-id pub-id-type="doi">10.1016/j.neunet.2014.09.003</pub-id> </citation>
</ref>
<ref id="B54">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Shaffer</surname>
<given-names>J.&#x20;B.</given-names>
</name>
<name>
<surname>Knezevic</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2010</year>). <article-title>Building Texture Evolution Networks for Deformation Processing of Polycrystalline Fcc Metals Using Spectral Approaches: Applications to Process Design for Targeted Performance</article-title>. <source>Int. J.&#x20;Plasticity</source> <volume>26</volume>, <fpage>1183</fpage>&#x2013;<lpage>1194</lpage>. <pub-id pub-id-type="doi">10.1016/j.ijplas.2010.03.010</pub-id> </citation>
</ref>
<ref id="B55">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Tan</surname>
<given-names>R. K.</given-names>
</name>
<name>
<surname>Zhang</surname>
<given-names>N. L.</given-names>
</name>
<name>
<surname>Ye</surname>
<given-names>W.</given-names>
</name>
</person-group> (<year>2020</year>). <article-title>A Deep Learning-Based Method for the Design of Microstructural Materials</article-title>. <source>Struct. Multidisc Optim</source> <volume>61</volume>, <fpage>1417</fpage>&#x2013;<lpage>1438</lpage>. <pub-id pub-id-type="doi">10.1007/s00158-019-02424-2</pub-id> </citation>
</ref>
<ref id="B56">
<citation citation-type="confproc">
<person-group person-group-type="author">
<name>
<surname>Tin Kam Ho</surname>
<given-names>T. K.</given-names>
</name>
</person-group> (<year>1995</year>). &#x201c;<article-title>Random Decision Forests</article-title>,&#x201d; in <conf-name>Proceedings of 3rd International Conference on Document Analysis and Recognition. Presented at the 3rd International Conference on Document Analysis and Recognition</conf-name> (<publisher-loc>Montreal, Que</publisher-loc>: <publisher-name>IEEE Comput. Soc. PressCanada</publisher-name>), <fpage>278</fpage>&#x2013;<lpage>282</lpage>. <pub-id pub-id-type="doi">10.1109/ICDAR.1995.598994</pub-id> </citation>
</ref>
<ref id="B57">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Torquato</surname>
<given-names>S.</given-names>
</name>
</person-group> (<year>2002</year>). <source>Random Heterogeneous Materials: Microstructure and Macroscopic Properties, Interdisciplinary Applied Mathematics</source>. <publisher-loc>New York</publisher-loc>: <publisher-name>Springer</publisher-name>. </citation>
</ref>
<ref id="B58">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Wargo</surname>
<given-names>E. A.</given-names>
</name>
<name>
<surname>Hanna</surname>
<given-names>A. C.</given-names>
</name>
<name>
<surname>&#xc7;e&#xe7;en</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
<name>
<surname>Kumbur</surname>
<given-names>E. C.</given-names>
</name>
</person-group> (<year>2012</year>). <article-title>Selection of Representative Volume Elements for Pore-Scale Analysis of Transport in Fuel Cell Materials</article-title>. <source>J.&#x20;Power Sourc.</source> <volume>197</volume>, <fpage>168</fpage>&#x2013;<lpage>179</lpage>. <pub-id pub-id-type="doi">10.1016/j.jpowsour.2011.09.035</pub-id> </citation>
</ref>
<ref id="B59">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Wu</surname>
<given-names>X.</given-names>
</name>
<name>
<surname>Proust</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Knezevic</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2007</year>). <article-title>Elastic-plastic Property Closures for Hexagonal Close-Packed Polycrystalline Metals Using First-Order Bounding Theories</article-title>. <source>Acta Materialia</source> <volume>55</volume>, <fpage>2729</fpage>&#x2013;<lpage>2737</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2006.12.010</pub-id> </citation>
</ref>
<ref id="B60">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Yabansu</surname>
<given-names>Y. C.</given-names>
</name>
<name>
<surname>Altschuh</surname>
<given-names>P.</given-names>
</name>
<name>
<surname>H&#xf6;tzer</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Selzer</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Nestler</surname>
<given-names>B.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
</person-group> (<year>2020</year>). <article-title>A Digital Workflow for Learning the Reduced-Order Structure-Property Linkages for Permeability of Porous Membranes</article-title>. <source>Acta Materialia</source> <volume>195</volume>, <fpage>668</fpage>&#x2013;<lpage>680</lpage>. <pub-id pub-id-type="doi">10.1016/j.actamat.2020.06.003</pub-id> </citation>
</ref>
<ref id="B61">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Yang</surname>
<given-names>Z.</given-names>
</name>
<name>
<surname>Yabansu</surname>
<given-names>Y. C.</given-names>
</name>
<name>
<surname>Al-Bahrani</surname>
<given-names>R.</given-names>
</name>
<name>
<surname>Liao</surname>
<given-names>W.-k.</given-names>
</name>
<name>
<surname>Choudhary</surname>
<given-names>A. N.</given-names>
</name>
<name>
<surname>Kalidindi</surname>
<given-names>S. R.</given-names>
</name>
<etal/>
</person-group> (<year>2018</year>). <article-title>Deep Learning Approaches for Mining Structure-Property Linkages in High Contrast Composites from Simulation Datasets</article-title>. <source>Comput. Mater. Sci.</source> <volume>151</volume>, <fpage>278</fpage>&#x2013;<lpage>287</lpage>. <pub-id pub-id-type="doi">10.1016/j.commatsci.2018.05.014</pub-id> </citation>
</ref>
<ref id="B62">
<citation citation-type="journal">
<person-group person-group-type="author">
<name>
<surname>Yilin</surname>
<given-names>G.</given-names>
</name>
<name>
<surname>Fuh Ying Hsi</surname>
<given-names>J.</given-names>
</name>
<name>
<surname>Wen Feng</surname>
<given-names>L.</given-names>
</name>
</person-group> (<year>2021</year>). <article-title>Multiscale Topology Optimisation with Nonparametric Microstructures Using Three-Dimensional Convolutional Neural Network (3D-CNN) Models</article-title>. <source>Virtual Phys. Prototyping</source> <volume>16</volume>, <fpage>306</fpage>&#x2013;<lpage>317</lpage>. <pub-id pub-id-type="doi">10.1080/17452759.2021.1913783</pub-id> </citation>
</ref>
<ref id="B63">
<citation citation-type="book">
<person-group person-group-type="author">
<name>
<surname>Zhang</surname>
<given-names>A.</given-names>
</name>
<name>
<surname>Lipton</surname>
<given-names>Z. C.</given-names>
</name>
<name>
<surname>Li</surname>
<given-names>M.</given-names>
</name>
<name>
<surname>Smola</surname>
<given-names>A. J.</given-names>
</name>
</person-group> (<year>2021</year>). <source>Dive into Deep Learning</source>. <comment>Preprint. arXiv:2106.11342&#x20;Cs</comment>. </citation>
</ref>
</ref-list>
</back>
</article>