For three hours, Bryant provided testimony against Los Angeles county for invasion of privacy over photos of her husband’s corpse
Vanessa Bryant testified on Friday that she was only beginning to grieve the loss of her husband, basketball star Kobe Bryant, and their 13-year-old daughter Gianna when she was faced with the fresh horror of learning that sheriff’s deputies and firefighters had shot and shared photos of their bodies at the site of the helicopter crash that killed them.
“I felt like I wanted to run, run down the block and scream,” she said, her tears turning to sobs and her voice quivering. “It was like the feeling of wanting to run down a pier and jump into the water. The problem is I can’t escape. I can’t escape my body.”
During her three hours on the witness stand in a Los Angeles federal court, where she is suing Los Angeles county for invasion of privacy over the pictures, Bryant said she had fought to get through both public and private memorials for her loved ones and seven others who were killed on 26 January 2020, and thought she was ready to really begin the grieving process about a month later.
She was with friends and her surviving daughters, and holding her seven-month-old baby, when she received a call about a Los Angeles Times story on the crash-site photos.
“I bolted out of the house and around to the side so my girls wouldn’t see,” she said. “I was blindsided again, devastated, hurt. I trusted them. I trusted them not to do these things.”
Evidence presented at trial showed that a sheriff’s deputy showed a photo of Bryant’s body to a bartender as he drank, spurring an official complaint from another man drinking nearby, and that firefighters shared them with each other at an awards banquet. Others shared them with spouses.
An attorney for the county said the photos had been taken only because they were essential for assessing the site moments after the crash, and that when LA county sheriff Alex Villanueva learned they were being shared, he demanded they all be deleted.
No photos emerged publicly, but Vanessa Bryant said she has constant worry that some still might.
“I live in fear every day of being on social media and these popping up,” she testified. “I live in fear of my daughters being on social media and these popping up.”
She said the thought keeps her awake at night as she lies next to her three-year-old and her five-year-old, and sometimes leads to panic attacks in which she can’t breathe.
Under cross-examination from J Mira Hashmall, the lawyer representing LA county at the trial, Bryant testified that she had not received any medical diagnosis of having had panic attacks, or any mental health disorder, nor had she taken any medications for them.
She said she had talked to a therapist for about 18 months after the crash, but had not since.
“I feel like sometimes it helps,” Bryant said, “but sometimes it’s completely draining.” Hashmall spent much of her 90-minute cross-examination going through the business roles Bryant now plays, including acting as president of her husband’s multimedia company, Granity Studios, overseeing the publication of one book he wrote and helping to finish and publish another, heading the foundation started for Kobe and Gianna, and establishing several other companies.
Hashmall suggested that Bryant’s ability to do all of this meant she was functioning well and was not overcome with fear and anxiety.
“It sounds like on top of everything else you’re juggling a business empire,” Hashmall said at one point.
“For me, it’s a labor of love,” said Bryant, who remained calm and composed during cross-examination.
She cried frequently, and laughed occasionally, during the questioning of her attorney Luis Li, who had her describe her life with her “proud girl-dad” husband and their daughters.
“He was just such a beautiful and devoted father,” she said.
Bryant chronicled the day of the crash, her anguish, and her frustration at trying to learn whether her husband and daughter were still alive after she initially heard from an assistant that there were five survivors.
She described Sheriff Villanueva coming into a room where she waited at Lost Hills sheriff’s station and confirming that her husband and daughter had been killed. He asked if there was anything he could do for her.
“I told him, if you can’t bring my babies back, then please secure the area,” Bryant said. “I’m concerned about paparazzi.”
“Did the sheriff tell you one of his deputies had already gone to the hill to take close-up pictures of crash victims?” Li asked.
“No,” Bryant responded.
During cross-examination, Hashmall said the deputy, Doug Johnson, who hiked through tough terrain into the hills in northern Los Angeles county to the crash site and shot the photos that were later shared, was only trying to use them to assess the situation.
“You can understand why he would want the same information you did,” Hashmall said.
“I don’t think you need to take close-up photos of people to determine how many people are on an aircraft,” Bryant replied. “I think he could have just counted.” Bryant’s side rested its case after her testimony, which came on the eighth day of the trial.
‘I live in fear’: Vanessa Bryant testifies at trial over Kobe’s crash images – The Guardian US
Artlist Launches Its First Video and Image Editing Software – Indie Film Hustle
Tired of NOT Making Money with Your Films? Want to Make a Living Making Movies?
Artlist, the creative technology company with innovative post-production software and over 1M digital assets available to license, announces today the introduction of the first software release under the FXhome by Artlist brand, an upgraded video, VFX, and image editing program with improvements and new subscription plans. FXhome was acquired in 2021 as part of Artlist’s mission to increase the value of its products. This represents the first and significant move to align the FXhome by Artlist brand with its parent company’s vision.
This new features and business model will make Artlist’s line of software products more usable and cost-effective than ever before and break down barriers for digital creators, which has been the company’s goal since it was founded. As part of the overhaul, Artlist has integrated part of its high-quality digital assets directly into the software to have it accessible to anyone using it and added four new subscription tiers for FXhome by Artlist’s products:
Artlist will continue to support and maintain. Those users can, of course, still upgrade to the new version of HitFilm to receive all the latest features and updates.
“We’ve been user-first since the beginning, and our decision to introduce this product upgrade is rooted in that. It was so important to us to do this in a way that stays true to users who prefer our existing model but expands affordability and accessibility,”
says Josh Davies, VP Software of Artlist and Founder of FXhome.
“HitFilm, CamTrackAR, Imerge, and the rest of the programs we provide are already tailor-made for creators. Now the purchasing model is too.”
“We’re excited to launch our very first Artlist release under the FXhome by Artlist brand, aligned under our signature revolutionary subscription-based model,”
says Ira Belsky, Co-Founder and Co-CEO of Artlist.
“We’ve seen a major paradigm shift towards subscription models throughout the content creation industry and the creator economy over the last few years due in large part to the subscription models we introduced and implemented years ago across Artlist’s asset catalogs. We know from experience that increasing product value and implementing flexible purchase plans are the best way to meet independent creators in all career levels. We want all of them to have access to these amazing offerings.”
To celebrate the launch, Artlist is offering special discount pricing for all paid tiers, with discounts up to 23% off:
Pro will be available at a $15.99 monthly subscription and a $119.99 annual subscription; then $19.99 monthly and $149.99 annually after the promotion period
Learn more about the new subscription tiers:
This is a sponsored post
Artlist is a leading creative technology company offering content creators powerful video and image editing software, and over 900,000 professional digital assets under a revolutionary, global license that covers every project worldwide.
In its mission to empower people to tell stories through video, Artlist offers four products: Artlist.io, a royalty-free music and SFX platform; Artgrid.io, a curated stock footage platform; MotionArray.com, an all-in-one marketplace for creators including video templates, presets, music, SFX, stock footage, motion graphics, and stock photos; and FXhome by Artlist, a cutting-edge video, VFX and image editing software. Artlist works with a growing team of contributors across the globe, updating its catalogs daily with fresh content while preserving a consistently high level of quality across its media and platforms.
Since its founding in 2016, Artlist has revolutionized the industry by offering subscription-based products under an unmatched license, and is becoming the ultimate 360-degree solution for content creators. Recognized as a top startup by LinkedIn and WIRED, Artlist counts Google, Apple, Nike, Coca-Cola, Ikea, Mercedes, Samsung, Wix, Netflix, Dior, and many more among its 17 million clients.
READ NEXT:
By Alex Ferrari
In this crash course film distribution expert Alex Ferrari shows you the top 5 distribution agreements and pitfalls to avoid, what a standard deal looks like, and much more.
Directing
Film Production
Cinematography
Screenwriting
Film Distribution
Post Production
Short Films
Documentary
Film History
Apple
Android
IFH Podcast
Bulletproof Screenwriting
The Directors Series
Filmmaking Motivation
Inside the Screenwriter’s Mind
IFH Podcast Network
Sponsors
IFH
Alex Ferrari
Charity
Free Classes
Script Coverage
Courses
Consulting
Filmmaking Resources
Books
Feature Films
Indie Film Hustle TV
IFH Academy
Privacy Policy
Terms of Use
DMCA Policy
Affiliate Disclaimer
Contact IFH
© 2022 IFH Industries, Inc. All Rights Reserved.
4 Reasons the Mac Studio and Studio Display Won't Replace the 27-Inch iMac – MUO – MakeUseOf
Apple discontinued the 27-inch iMac with the launch of the Mac Studio and Studio Display, but we think they make for terrible replacements.
Apple’s Mac Studio and Studio Display have been since early 2022, and the combination is supposed to be a good choice for people who want a powerful desktop but don’t want to spend $6,000 on a Mac Pro. The Mac Studio is a more than capable computer towers, and it pairs with an excellent 27-inch display.
Even though the Mac Studio and Studio Display are great products, they may not be the right choice for people who were hoping for Apple to release a new 27-inch iMac instead.
When the Mac Studio and Studio Display were released, Apple stopped selling the 27-inch iMac, leaving the 24-inch M1 iMac the only option for consumers who want an all-in-one Apple desktop. Some might consider the Mac Studio and Studio Display as a replacement for the 27-inch iMac since the Studio Display shares basically the same screen as the outgoing 5K iMac. But if you look at the whole package, it isn’t a perfect fit.
There is also the M1 Mac mini. However, consumers who purchase iMacs benefit from the all-in-one package, so the Mac mini is similar to the Mac Studio in that it isn’t an ideal alternative either.
Furthermore, those alternatives aren’t as upgradeable as the 27-inch iMac used to be. You can only get 16GB of ram in the M1 iMac and the Mac mini, while the 27-inch iMac use to max out at 128GB. That memory was also user upgradable in the 27-inch iMac.
There is a gap in the iMac lineup by not having a powerful, 27-inch desktop Mac, and here are all the reasons we think the Mac Studio and Studio Display aren’t good replacements.
To break down the prices, the Studio Display starts at $1,699 if you want the standard glass and a non-height-adjustable stand. You will have to pay extra for the nano-texture option or a different stand. The Mac Studio starts at $1,999 and comes with an M1 Max chip, 32GB of unified memory, and 512GB of storage. The total for a base Studio Display and a base Mac Studio is nearly $3,700, before taxes.
That is substantially higher than the $1,799 the 27-inch iMac used to start at in the lineup. This can price out specific consumers who want a larger Mac but can’t afford to spend nearly $4,000. Even if you pair the Studio Display with an M1 Mac mini, you are still looking at almost $2,400. This is not a small price increase when compared to the 27-inch iMac.
Apple’s 27-inch iMac was a good balance between price and power. While the Mac Studio is significantly more powerful than the 27-inch iMac, not everyone is looking for a desktop that has that power.
In addition to that, the 27-inch iMac was also user upgradeable. If you needed to add more RAM down the road, the 27-inch iMac included a small compartment at the back of the computer where you could add additional memory sticks. Apple silicon Macs are not user upgradeable in this way.
One big reason why people loved the 27-inch iMac was that it was an all-in-one computer. It came with the built-in display, the computer, and a keyboard and mouse for $1,799. Since the Mac Studio is a modular system, you must purchase the display, keyboard, and mouse separately.
These purchases start to add up and can be expensive, certainly more than the base price of the 27-inch iMac. Also, many monitors don’t come with built-in speakers, camera, or native macOS support. You don’t have to worry about this when you buy the iMac, but with the Mac Studio you’ll need to research a lot more purchases.
AppleCare for an iMac costs $169. However, if you want to purchase AppleCare for a Mac Studio and Studio Display, they are two separate purchases at a total cost of nearly $320. AppleCare could be worth it if you depend on your computer for work and won’t be able to afford a replacement. But at those costs, it really starts to tip the balance.
Apple’s Mac Studio and Studio Display are good products, but they are not the perfect replacement for the 27-inch iMac. Apple’s Mac desktop lineup is missing the larger all-in-one machine, and it’s created a divisive situation for users who want a larger screen desktop.
The Mac Studio and Studio Display are great if you need a powerful machine for creative work, but if you’re looking for a family computer with a large display, you should definitely look elsewhere.
Will is a Freelance Writer at MUO. He has had a passion for technology since he was young and has focused his career on the tech industry. When he’s not working, he enjoys spending time with friends and family, working out, and keeping up with new and emerging technologies.
Join our newsletter for tech tips, reviews, free ebooks, and exclusive deals!
Reconfigurable electro-optical logic gates using a 2-layer multilayer perceptron | Scientific Reports – Nature.com
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
Advertisement
Scientific Reports volume 12, Article number: 14203 (2022)
Metrics details
In this work, we aim to use the optical amplifiers, directional couplers and phase modulators to build the electro-optical gates. Thanks to the 2-layer-multilayer-perceptron structure, the inversion of matrix is performed to obtain the coupling ratio of the directional couplers and the phase delay of the phase modulators. The electro-optical OR, AND, XOR, NAND, NOR and XNOR gates are demonstrated. Moreover, we not only study the results under the ideal condition of device, but also discuss the imperfect situation with 1% error of fabrication or operation to study the tolerance of this system. Through our simulation results, the visibility of the gate output can be higher than 0.83. The gates can be fabricated in a silicon-based chip to develop the integrated optics computing system.
Optical computing is a striking issue in recent years. The logic gates are the fundamental devices to carry out this task1. All-optical logic gates based on photonic crystal (PC) waveguides, and ring resonators were proposed recently2,3,4. Also, there are many all-optical structures proposed to demonstrate the all-optical reservoir computing devices5,6,7,8. Thanks to the progress of the photonic technology which have been investigated intensively in recent decades, we can make these devices as a tiny chip9,10,11,12. Moreover, the temporal solitons and the continuous-wave light sources were used as the carriers of the input and output signals. The refractive index change of the waveguide by means of non-linear Kerr effect with high power light beam produces the tuning or switching of the output states of logic gates13,14. However, a long propagation path around several micrometers or even millimeters through the dielectric waveguides to deliver the light from light source to the logic gates might not be avoidable in a dielectric material (such as silicon) chip. The light might be absorbed by the waveguides leading to the deformation of the solitons. The low absorption and dispersion waveguides such as the hollow waveguide or using 3-dimensional PC structure over the waveguide might be considered to connect the light sources and the logic gates as well as to connect between the logic gates15,16,17.
Active components such as a tri-mode laser, Fabry–Perot laser, and semiconductor saturable absorber had also been proposed to form the all-optical logic gates14,18,19. Through tuning the optical injection of the tri-mode lasers, the change of logical state can be achieved14. High-speed all-optical logic gates at the bit rate of 340 Gb/s using the cross-gain modulation in semiconductor optical amplifier has also been achieved19.
Recently, researchers proposed the logic gates which are formed by microring/microdisk modulator20,21. Although the microring-/microdisk-modulators are tinier than our logic gate, the control process is more complicated than our design. With regard to the microring-modulator logic gates, they can achieve only AND, OR and XOR gates20. However, they need to modify the optical structure to carry out the corresponding function. In our study, we only need to tune the coupling ratio of the directional couplers and the phase of the phase shifter to achieve 6 logic gates at the single output port. For microdisk-modulator logic gates, the authors also can achieve OR, NOR, XNOR, XOR, AND and NAND logic gates21. However, the tunable laser providing 3 different wavelengths should be used. The size of the laser source and the external cavity to tune the wavelength might occupy several hundred square micrometers or even several square millimeters. The dimension of the integrated chip could be similar to that of our design.
In our previous work, we took advantage of the reservoir computing (RC) and unsupervised learning method optical neuron networks to establish an XOR gate5. This XOR gate with low bit error ratio was obtained by scanning the delay of optical phase shifter. In the present work, an electro-optical (EO) logic gates based on RC without the recurrent property are proposed. Obviously, it is a 2-layer-multilayer-perceptron (2-layer MLP) structure. This logic gates are composed of the optical amplifiers, directional couplers (DCs), and phase modulators (PMs). Comparing this new logic gates to our previous works, the feedback signal (the recurrent property) is removed in this new design. The switching time between 0 and 1 for the output of the gates can be shortened. In this work, the matrix inversion is performed to calculate the coupling ratio of the DCs and the phase delay of the PMs using the target output of the logic gates in the same optical configuration. The reconfigurable electro-optical OR, AND, XOR, NAND, NOR and XNOR gates formed by one optical structure are demonstrated.
It is known that RC system has been extensively used in the development of logic gates, waveform recognition, and rainfall prediction5,22,23,24. This system is composed of the input weight matrix ({W}^{in}), the recurrent weight matrix (W), and the readout matrix ({W}^{out}). The input matrix ({W}^{in}) is used to scale the size of the input data to the size of the matrix W. The value of the elements in Win and W is chosen randomly. In this study, the learning is completed in a single pass-through training data. The optimal readout matrix ({W}^{out}) is used to generate the output of the RC system.
In RC system, the neuron can be described as a function of the current input and its previous calculation result which can be expressed by25
where the function f is the nonlinear activation function of the neuron. Usually, the hyperbolic tangent function, tanh(), is used as the nonlinear activation function to converge the output value of neuron within − 1 and 126. In this research, the non-linear function of our optical neuron is provided by the optical amplifiers. (The optical setup is shown in Fig. 1.) (L) is the leaky rate which is a real number from 0 to 1. However, since we do not use the recurrent property of RC in the optical neural network, L and W are set to be 1 and null, respectively producing a 2-layer MLP structure. (uleft(nright)) represents the n-th input data. Thus, the reservoir activation state, (Xleft(nright)), is modified to (fleft({{varvec{W}}}^{in}uleft(nright)right)). The network target output ({Y}^{target}) is given by
The optical setup of this EO logic gate. CW, DC and PM stand for the continuous wave, the 2 by 2 directional coupler, and the phase modulator. The possible recombination of the input signals A and B is (0, 0), (0, 1 V), (1 V, 0) and (1 V, 1 V). The corresponding Ytarget for the XOR gate is [0 1 1 0].
In Eq. (3), M and N are the number of the neurons and the number of the input data, respectively. By collecting the training data (X) and the target training signals ({Y}^{target}), the readout matrix ({W}^{out}) can be obtained by inversing the matrix X from Eq. (2).
The main feature of 2-layer MLP is the fact that the training (or the optimization) is done by dealing with the readout matrix ({W}^{out}) using the inverse of the matrix X. As we mentioned above, once we obtain the elements of ({W}^{out}), we could deduce the coupling ratio and phase in our DCs and PMs, respectively. This will be detailed in the next section. Through this method, we can obtain the parameters of the optical devices in this optical neural network.
Figure 1 illustrates the optical setup of our EO neural-network-based logic gates. The CW laser source at the wavelength of 1550 nm with the power of 0.1 W is launched into the DCin of our logic gate. The optical nonlinear effect of the optical fibers which connect the optical devices are ignored. This DCin with the coupling ratio of 50% is used to divide equally the input light into two light beams. The two light beams are launched into the two phase modulators (PMA and PMB in Fig. 1) individually. The half-wave voltage Vπ of the PMA and PMB is set randomly to be 1 V and 2 V, respectively. The input signals A and B are applied on the phase modulators PMA and PMB, respectively. The applied voltages are 0 or 1 V of the phase modulators to present the logic 0 and 1, respectively. PMA and PMB are connected to the directional coupler DCA. After that, the output signals of DCA are launched into DCB and DCC. DCA, DCB and DCC serve as the input matrix Win of the 2-layer MLP in which the values of the elements are chosen randomly. The coupling ratio of DCA, DCB and DCC is therefore set randomly. In our research, for the example of the XOR gate, the coupling ratios of the DCA, DCB and DCC are set randomly to be 0.56, 0.83 and 0.96, respectively.
The four outputs of the DCB and DCC are connected to the optical amplifiers A1, A2, A3 and A4, individually. The optical gain of the amplifiers is given by Go/(1 + G0P/Psat) where G0, P, Psat are the small signal power gain, the power of the input light, and the saturation output power, respectively. For all amplifiers, the small signal power gain and the saturation output power are set to be 30 dB and 30dBm, respectively. The optical amplifiers provide the non-linear function which is required for 2-layer MLP system. The complex electric field of the output light of the amplifiers A1, A2, A3 and A4 are denoted by X1, X2, X3 and X4, respectively, which are regarded as the output of the neurons. Therefore, the number of the neurons, M in Eqs. (3) and (4) is 4. In other words, an optical amplifier can be regarded as an optical neuron. The readout matrix ({W}^{out}) is composed of the 3 phase modulators PM1, PM2 and PM3 as well as the 3 directional couplers DC1, DC2 and DC3. ({varnothing }_{1}), ({varnothing }_{2}) and ({varnothing }_{3}) are the phase delay of the phase modulators PM1, PM2 and PM3, respectively. ({alpha }_{1}), ({alpha }_{2}) and ({alpha }_{3}) are the coupling ratios of the DC1, DC2 and DC3, respectively. The general formula of the output of electric field for the phase modulator and the directional coupler can be given by Eqs. (6) and (7), respectively.
where ({X}_{in}) and ({X}_{out}) represent the complex input and output electric fields of the components, respectively. (varnothing) and (alpha) are the phase delay of the phase modulators and the coupling ratio of the 2 by 2 DCs, respectively. The two input lights in the directional coupler are interfered. Using Eqs. (6) and (7), the complex electric field of the output light of the whole system ({Y}^{target}) can be derived by
The first step of the design process is to define the gate. For example, for an XOR gate, the possible recombination of the input signals A and B are (0, 0), (0, 1 V), (1 V, 0) and (1 V, 1 V). Thus, in Eq. (4), the number of the possible recombination of the input signals, (N), is 4. The corresponding result ({Y}^{target}) for the XOR gate are [0 1 1 0]. The complex electric fields X1, X2, X3 and X4 are located at the output of A1, A2, A3 and A4, respectively, as shown in Fig. 1. For each recombination of the input signals, the complex values of X1, X2, X3 and X4 are collected to form the 4 by 4 matrix X of Eq. (4). According to Eq. (5), the values of ({w}_{1}), ({w}_{2}), ({w}_{3}) and ({w}_{4}) of the 1 by 4 matrix ({W}^{out}) can be obtained using Ytarget and X. The coefficients of the terms X1, X2, X3 and X4 in Eq. (8) are equal to the 4 complex values ({w}_{1}), ({w}_{2}), ({w}_{3}) and ({w}_{4}) of ({W}^{out}) forming 4 simultaneous equations. The phases of the complex numbers ({w}_{1}), ({w}_{2}), ({w}_{3}) and ({w}_{4}) are denoted by ({theta }_{1}), ({theta }_{2}), ({theta }_{3}) and ({theta }_{4}). The boundary conditions for ({alpha }_{1}), ({alpha }_{2}), ({alpha }_{3}), are real numbers between 0 and 1. By solving the simultaneous equations, ({alpha }_{1}), ({alpha }_{2}), ({alpha }_{3}), ({varnothing }_{1}), ({varnothing }_{2}) and ({varnothing }_{3}) can be obtained by
The ({Y}^{target}) for the OR, AND, XOR, NAND, NOR, XNOR logic gates is [0 1 1 1], [0 0 0 1], [0 1 1 0], [1 1 1 0], [1 0 0 0] and [1 0 0 1], respectively. For each gate, the coupling ratios of the DCA, DCB and DCC in Fig. 1, αA, αB and αC, are chosen randomly for 100 times to obtain the best tolerance. The detail calculation of tolerance will be discussed in the next section. The obtained αA, αB and αC for each gate are listed in Table 1. The corresponding coupling ratio of the directional couplers DC1, DC2, DC3 and the phase delay of the phase modulators PM1, PM2, PM3 calculated through Eqs. (2)–(14) are listed in Table 2. The output powers of the OR, AND, XOR, NAND, NOR and XNOR logic gates for different input signals A and B are shown in Fig. 2. For the electro-optical logic gates, the relation between the input signals and the output of the 2-layer MLP system follows the truth table of the corresponding logic gates. The reconfigurable logic gate can be achieved by tuning the parameters of the directional couplers27,28,29,30 and phase modulators31 as listed in Tables 1 and 2. It is worth mentioning that the phase delay of the phase modulators for NOR gate is close to null revealing that the phase modulators PM1, PM2 and PM3 might be removed. For the other gates, it is might be possible to find the solution to remove the phase modulators.
The output power of (a) OR, (b) AND, (c) XOR, (d) NAND, (e) NOR and (f) XNOR logic gates.
We use the visibility of the output signals of logic 0 and logic 1 to evaluate the tolerance of our logic gates. The definition of visibility is shown in Eq. (15)32
where ({I}_{min}^{1}) and ({I}_{max}^{0}) are the minimum light intensity of the high level (logic 1) and the maximum light intensity of the low level (logic 0), respectively. For example, for the output logic of XOR gate [0 1 1 0], if the output intensity of the 2-layer MLP system is [0.001, 0.013, 0.011, 0.002], we choose ({I}_{min}^{1}) and ({I}_{max}^{0}) as 0.011 and 0.002, respectively to calculate the visibility.
Since the coupling ratio of the directional couplers DC1, DC2, DC3 and the phase delay of the phase modulators PM1, PM2, PM3 are solved using Eqs. (2)–(14), the light intensity of the low level (logic 0) is null. The corresponding visibility of the output signals for all logic gates is unity.
During the fabrication of the directional couplers by optical fibers or by silicon waveguides, 1% error of the coupling ratio may happen. During the operation of the phase modulators by applying the voltage on the device, 1% error of the phase delay may occur. We calculate the visibility of the output signals as the coupling ratio of DC1, DC2, DC3 and the phase delay of the phase modulators PM1, PM2, PM3 are increased with 1% error. For example, for XOR gate, α1 obtained from Eq. (9) is 49% as listed in Table 2. As α1 is increased from 49 to 50% (αA, αB, αC, α2, α3, ϕ1, ϕ2 and ϕ3 are left unchanged as listed in Tables 1 and 2), the visibility of the output signal is calculated to be 0.99. For another example, for the XOR gate, as ϕ1 is increased from 8.88° to 12.48° (increasing 3.6° = 1% from – π to π), the visibility of the output signals is 0.91. For 100 calculations of different αA, αB, αC of each gates, the highest visibility of the 2-layer MLP system for all logic gates due to 1% error of the devices is listed in Table 3.
We can observe that for 1% error of the coupling ratio of the directional couplers DC1, DC2 and DC3, the worst visibility is 0.98. For 1% error of the phase delay of the phase modulators PM1, PM2 and PM3, the worst visibility is 0.83. It seems that the results could be acceptable showing that the reconfigurable optical logic gates can be achieved even with the fabrication or operation errors. In electronics, a flip-flop or latch is a circuit that can be used to store state information with a pair of cross-coupled NOR of NAND gates. The further application of the reconfigurable optical logic gates is to build the flip-flop. The optical memory could be realized.
In this work, we successfully establish an optical structure which can achieve EO logic gates. Thanks to this 2-layer-MLP structure, we can carry out the OR, AND, XOR, NAND, NOR and XNOR logic gates. In other words, we could take advantage of one optical neural network to achieve the most basic logic gates in optical approach. According to Eq. (5), we obtain the coupling ratio of the directional couplers DC1, DC2, and DC3 and the phase delay of the phase modulators PM1, PM2, PM3 under the ideal condition. The tolerance of the optical devices in our optical neural network is studied. The visibility changes of the output signals due to the 1% error during the fabrication and operation of the optical devices could be acceptable indicating that the reconfigurable optical logic gates can be realized in one optical configuration. Comparing our EO logic gates to the conventional electric logic gates, this EO logic gates have the capability of higher operating frequency, and high speed. In our study, the logic-gates functions among OR, AND, XOR, NAND, NOR and XNOR can be designed when the PMs and the coupling ratio of DCs are tuned in this EO logic gates. Thus, the control process is simpler. The present design uses the neural networks without recurrent property. The calculation performance could be faster than that of the previous work5. The optical logic gates can be applied to build an optical memory using the latch structure to store the information.
The datasets used and/or analysed during the current study available from the corresponding author on reasonable request.
Saharia, A. et al. A comparative study of various all-optical logic. (ed. Janyani, V., Singh, G., Tiwari, M., & d’Alessandro, A.) 429–437 (Springer, 2019).
Selvakumari, M., Prabha, K. R., Robinson, S. & Batcha, A. S. Investigation on two dimensional photonic crystal based all optical logic gates. (ed. Zakaria, Z.& Emamian, S.S.) 129–142 (Springer, 2022).
Heydarian, K., Nosratpour, A. & Razaghi, M. Design and analysis of an all-optical NAND logic gate using a photonic crystal semiconductor optical amplifier based on the Mach–Zehnder interferometer structure. Photonics Nanostruct. Fundam. Appl. 49, 100992 (2022).
Article Google Scholar
El Haffar, R., Mahboub, O., Farkhsi, A. & Figuigue, M. All-optical logic gates using a plasmonic MIM waveguide and elliptical ring resonator. Plasmonics 17, 831–842 (2022).
Article Google Scholar
Lin, C. E. et al. All optical XOR logic gate formed by unsupervised optical neuron networks. Neurocomputing 460, 205–210 (2021).
Article Google Scholar
Duport, F., Schneider, B., Smerieri, A., Haelterman, M. & Massar, S. All-optical reservoir computing. Opt. Express 20, 22783–22795 (2012).
ADS Article Google Scholar
Dejonckheere, A. et al. All-optical reservoir computer based on saturation of absorption. Opt. Express 22, 10868–10881 (2014).
ADS Article Google Scholar
Feldmann, J., Youngblood, N., Wright, C. D., Bhaskaran, H. & Pernice, W. H. P. All-optical spiking neurosynaptic networks with self-learning capabilities. Nature 569, 208–214 (2019).
ADS CAS Article Google Scholar
Cai, D. P. et al. Compact pulley-type microring resonator with high quality factor. Appl. Phys. Express 7, 112202 (2014).
ADS Article Google Scholar
Cai, D. P. et al. High Q-factor microring resonator wrapped by the curved waveguide. Sci. Rep. 5, 10078 (2015).
ADS Article Google Scholar
Chiu, W. Y. et al. A photonic crystal ring resonator formed by SOI nano-rods. Opt. Express 15, 15500–15506 (2007).
ADS CAS Article Google Scholar
Chen, C. C., Pertsch, T., Iliew, R., Lederer, F. & Tunnermann, A. Directional emission from photonic crystal waveguides. Opt. Express 14, 2423–2428 (2006).
ADS Article Google Scholar
Jandieri, V. et al. Functional all-optical logic gates for true time-domain signal processing in nonlinear photonic crystal waveguides. Opt. Express 28, 18317–18331 (2020).
ADS Article Google Scholar
Liu, J. C. et al. Numerical simulation of all-optical logic gates based on hybrid-cavity semiconductor lasers. J. Opt. Soc. Am. A 38, 808–816 (2021).
ADS Article Google Scholar
Lo, S. S., Wang, M. S. & Chen, C. C. Semiconductor hollow optical waveguides formed by omni-directional reflectors. Opt. Express 12, 6589–6593 (2004).
ADS CAS Article Google Scholar
Chiu, H. K., Hsiao, F. L., Chan, C. H. & Chen, C. C. Compact and low-loss bent hollow waveguides with distributed Bragg reflector. Opt. Express 16, 15069–15073 (2008).
ADS CAS Article Google Scholar
Chen, C. C., Tsai, Y. L., Hsu, C. L. & Chang, J. Y. Propagation loss reduction of photonic crystal slab waveguides by microspheres. Opt. Express 12, 3934–3939 (2004).
ADS CAS Article Google Scholar
Nakarmi, B., Zhang, X. & Won, Y. H. All-optical 4×10 Gbps NAND gate using single mode Fabry–Pérot laser diode. Opt. Express 23, 26952–26961 (2015).
ADS CAS Article Google Scholar
Pradhan, R. All-optical XNOR/NOT logic gates and LATCH based on a reflective vertical cavity semiconductor saturable absorber. Appl. Opt. 53, 3807–3813 (2014).
ADS Article Google Scholar
Ying, Z. et al. Automated logic synthesis for electro-optic logic-based integrated optical computing. Opt. Express 26, 28002–28012 (2018).
ADS Article Google Scholar
Gostimirovic, D. & Ye, W. N. Ultracompact CMOS-compatible optical logic using carrier depletion in microdisk resonators. Sci. Rep. 7, 1–10 (2017).
CAS Article Google Scholar
Cheng, T. Y., Chou, D. Y., Liu, C. C., Chang, Y. J. & Chen, C.-C. Optical neural networks based on optical fiber-communication system. Neurocomputing 364, 239–244 (2019).
Article Google Scholar
Cheng, T. Y., Liu, C. C., Jhou, D. Y. & Chen, C. C. Impact of coupling topology upon noise robustness of small optical reservoirs. Sci. Rep. 10, 14086 (2020).
ADS CAS Article Google Scholar
Yen, M. H., Liu, D. W., Hsin, Y. C., Lin, C. E. & Chen, C. C. Application of the deep learning for the prediction of rainfall in Southern Taiwan. Sci. Rep. 9, 12774 (2019).
ADS Article Google Scholar
Lukoševičius, M. A Practical Guide to Applying Echo State Networks (ed. Montavon, G., Orr, G. B. & Müller, K. R.) (Springer, 2012).
Lukoševičius, M., Jaeger, H. & Schrauwen, B. Reservoir computing trends. KI Künstliche Intelligenz 26, 365–371 (2012).
Article Google Scholar
Orlandi, P. et al. Tunable silicon photonics directional coupler driven by a transverse temperature gradient. Opt. Lett. 38(6), 863–865. https://doi.org/10.1364/OL.38.000863 (2013).
ADS CAS Article PubMed Google Scholar
Zhang, Q. et al. Reconfigurable directional coupler in lithium niobate crystal fabricated by three-dimensional femtosecond laser focal field engineering. Photon. Res. 7, 503–507 (2019).
CAS Article Google Scholar
Chen, C. C., Porte, H., Goedgebuer, J. P., Armbruster, V. & Ferriere, R. Adjustment of the coupling ratio by laser ablation of Ti:LiNbO3 directional couplers. Opt. Laser Technol. 34, 453–455 (2002).
ADS CAS Article Google Scholar
Lo, S. S. & Chen, C. C. High finesse of optical filter by a set Fabry–Perot cavity. J. Opt. Soc. Am. B 24, 1853–1856 (2007).
ADS CAS Article Google Scholar
Sorianello, V., Midrio, M. & Romagnoli, M. Graphene on SOI phase modulation. In 2015 Fotonica AEIT Italian Conference on Photonics Technologies. 1–3. https://doi.org/10.1049/cp.2015.0133 (2015).
Bahaa, M. C. T. & Saleh, E. A. Fundamentals of Photonics 2nd edn, 419–420 (Wiley, Hoboken, 2007).
Google Scholar
Download references
The authors thank Taiwan Semiconductor Research Institute for the software support. This work was supported by Ministry of Science and Technology, Taiwan [Grant Numbers MOST 110-2112-M-008-032].
Department of Electronic Engineering, National Chin-Yi University of Technology, Taichung, 411030, Taiwan
Chu-En Lin
Department of Optics and Photonics, National Central University, Zhongli, 320317, Taiwan
Yueh-Heng Lu, Meng-Ting Zhou & Chii-Chang Chen
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
C.-C.C. supervised this research topic. C.-E.L., Y.-H.L. and M.-T.Z. executed the simulation. Y.-H.L. and M.-T.Z. performed the simulation result. C.-C.C. and C.-E.L. analyzed the results and wrote this manuscript. All authors discussed and commented on the results and manuscript.
Correspondence to Chii-Chang Chen.
The authors declare no competing interests.
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/.
Reprints and Permissions
Lin, CE., Lu, YH., Zhou, MT. et al. Reconfigurable electro-optical logic gates using a 2-layer multilayer perceptron. Sci Rep 12, 14203 (2022). https://doi.org/10.1038/s41598-022-18408-0
Download citation
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41598-022-18408-0
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
By submitting a comment you agree to abide by our Terms and Community Guidelines. If you find something abusive or that does not comply with our terms or guidelines please flag it as inappropriate.
Advertisement
Advanced search
© 2022 Springer Nature Limited
Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.
Russia-Ukraine war: at least 12 Russians killed in strike on Nova Kakhovka base, says Ukraine – as it happened – The Guardian
Footage on Telegram showed numerous burnt out trucks, collapsed buildings, and debris. This blog is now closed
At least 12 Russians have been killed in a strike on a base in the occupied city of Nova Kakhovka, according to the Ukrainian military.
Footage posted to Telegram by the State Border Guard Service showed numerous burnt-out trucks, collapsed buildings, and debris.
“The base of the occupiers was destroyed in Nova Kakhovka,” the post read. “At least 12 Rashists [supporters of Russian militarism] were liquidated.”
It’s just after 9pm in Ukraine. Here’s a round-up of all the latest from the conflict.
The Ukrainian military said at least 12 Russian were killed in a strike on a base in the occupied city of Nova Kakhovka. Footage posted to Telegram by the State Border Guard Service showed numerous burnt-out trucks, collapsed buildings, and debris.
Also in occupied southern Ukraine, the mayor of the city of Melitopol said explosions were heard near a Russian command centre. Ivan Fedorov said that a “loud explosion rang out near the enemy’s lair” in the centre of the city.
The top official in Russian-annexed Crimea has claimed the FSB security service has broken up what he described as a six-person terrorist cell of a banned Islamist group. It followed a series of strikes on Russian bases and other infrastructure across the region which Russia has sought to blame on “sabotage”.
Ukrainian President Volodymyr Zelenskiy has told Ukrainians living in Crimea to avoid Russia military bases following the strikes. “Do not approach the military objects of the Russian army,” he said in his latest video address, adding that they might also explode because of “bungling”.
Ukrainian defence minister Oleksii Reznikov has said that, contrary to Russian claims, Ukraine has not lost any of the Himars rocket launchers provided to it by the US. Since the start of the conflict, Ukraine has received at least 20 of the US-made launchers, and has used them to attack Russian ammunition depots, command posts, and air defences. Russia has repeatedly claimed to have destroyed a number of the systems.
The mayor of the village of Verkhnyo Rogachytsk in the Kherson region has been kidnapped, according to the deputy chair of the regional council. Yuri Sobolevsky said Svitlana Ivanivna was taken from her home at around 11am on Tuesday.
A Russian soldier has become one of just a handful to speak out publicly against the war in Ukraine. Speaking to the Guardian, Pavel Filatyev, who fled Russia this week after publishing a 141-page account of his time fighting as part of a paratrooper unit, said: “I don’t see justice in this war. I don’t see truth here.”
Four more vessels carrying agricultural products have sailed from ports in Ukraine. Turkish state media said the ships departed Odesa and Chernomorsk on Wednesday morning carrying sunflower meal, sunflower oil and corn, it said. They are the latest vessels to leave Ukraine under the terms of a grain export deal agreed between the UN, Ukraine, Turkey and Russia last month.
UN Secretary General Antonio Guterres is in Lviv to review the deal in talks with the Ukrainian and Turkish presidents. The three will also discuss “the need for a political solution to this conflict” and the situation at the Russian-controlled Zaporizhzhia nuclear power plant, a UN spokesperson said.
Polish Deputy Foreign Minister Marcin Przydacz has criticised German Chancellor Olaf Scholz for his opposition to a proposal to ban all Russian nationals from the European Union. Asked about the comments, Przydacz said: “[Scholz] did not read all those reports about the enormous support for Vladimir Putin’s policy among ordinary Russians, maybe he did not see all the videos in which people gather with the letter Z.”
Russia has appointed a new commander for its Black Sea fleet, according to Russian state media. Vice-Admiral Viktor Sokolov is said to have been presented to a meeting of the fleet’s most senior figures in the Sevastopol in Crimea. The previous commander of the fleet, Igor Osipov, is thought to have been removed following the sinking of its flagship, the Moskva, in April.
A Russian soldier has become one of just a handful to speak out publicly against the war in Ukraine.
Speaking to the Guardian, Pavel Filatyev said: “I don’t see justice in this war. I don’t see truth here.”
Filatyev fled Russia earlier this week after publishing a 141-page document providing a day-by-day account of his time fighting as part of a paratrooper unit.
He was eventually evacuated from the conflict after being wounded and contracting an eye infection. He says that, by then, he had decided he needed to expose the rot at the core of the invasion.
“We were sitting under artillery fire by Mykolaiv,” he said.
“At that point I already thought that we’re just out here doing bullshit, what the fuck do we need this war for? And I really had this thought: ‘God, if I survive, then I’ll do everything that I can to stop this.’”
Read the full story here.
A key advisor to Ukrainian President Volodymyr Zelenskiy has repeated a call for the Crimean Bridge to be destroyed.
Writing on Twitter, Mykhailo Podolyak said Ukraine had never granted permission for the construction of the bridge and that it was an “illegal object”.
“It harms the peninsula’s ecology and therefore must be dismantled. Not important how – voluntary or not,” he said.
The Crimean Bridge, built following Russia’s invasion of Crimea in 2014, spans the Kerch Strait, connecting the peninsula to mainland Russia.
The bridge was closed overnight on Tuesday following a series of strikes on Russian bases and other infrastructure in Crimea. The closure is thought to have come amid fears that it would also be targeted.
Speaking to the Guardian this week, Podolyak signalled that Ukraine regarded the bridge as a legitimate military target.
“It’s an illegal construction and the main gateway to supply the Russian army in Crimea. Such objects should be destroyed,” he said.
About "Kerchen bridge". As recorded in international law: Crimea – 🇺🇦. So this bridge is an illegal object, permission for the construction of which was not given by Ukraine. It harms the peninsula’s ecology and therefore must be dismantled. Not important how – voluntary or not.
Poland’s deputy foreign minister has criticised German Chancellor Olaf Scholz for his opposition to a proposal to ban all Russian nationals from the European Union.
In a press conference in Olso on Monday, Scholz said that this was “not the war of the Russian people, but it is Putin’s war”.
Asked about the comments during an appearance on Polish state broadcaster TVP1, Marcin Przydacz said he rejected the idea of “good Russians and a bad tsar”.
“[Scholz] did not read all those reports about the enormous support for Vladimir Putin’s policy among ordinary Russians, maybe he did not see all the videos in which people gather with the letter Z,” he said.
“Russian society, unfortunately, fed by propaganda for not even the last years, but for decades, hundreds of years, lives in an imperialist dream in which its position would depend on how many territories it seizes, how many countries it conquers.”
Ukrainian President Volodymyr Zelenskiy has told Ukrainians living in Crimea to avoid Russia military bases after three strikes on the peninsula in the past week.
“Do not approach the military objects of the Russian army,” he said in his latest video address, adding that they might also explode because of “bungling”.
Russia has blamed the explosions on “sabotage”, and earlier claimed to have detained six members of an Islamist terror cell.
Ukraine has not claimed responsibility for the attacks, but speaking about them to the Guardian a key adviser to the president said that “Crimea occupied by Russians is about warehouse explosions and high risk of death for invaders and thieves”.
Read the full story here.
UN Secretary General Antonio Guterres is in Lviv ahead of talks with the Ukrainian and Turkish presidents on Thursday.
A photo showing Guterres arriving in the city was posted to Twitter by an official UN account.
Guterres is set to meet with Volodymyr Zelenskiy and Tayyip Erdoğan to review the grain export deal agreed between the UN, Ukraine, Turkey, and Russia last month.
The three will also discuss “the need for a political solution to this conflict” and the situation at the Russian-controlled Zaporizhzhia nuclear power plant, a UN spokesperson said previously.
.@antonioguterres arrived Lviv where, tomorrow, he will meet @ZelenskyyUa & @RTErdogan. pic.twitter.com/PC1tBbXOF6
Russia has appointed a new commander for its Black Sea fleet, according to state media agency RIA Novosti.
The agency said Vice-Admiral Viktor Sokolov had been presented to a meeting of the fleet’s most senior figures in the Sevastopol in Crimea.
It added there was no public event because of what it called the “yellow level of terrorist threat” in the city.
The previous commander of the fleet, Igor Osipov, is thought to have been removed following the sinking of its flagship, the Moskva, in April.
Solokov previously served as the head of Saint Petersburg’s Kuznetsov Naval Academy, the main staff college for the Russian Navy.
Ukraine has not lost any of the Himars rocket launchers provided to it by the US, the Ukrainian defence minister has said.
Since the start of the conflict, Ukraine has received at least 20 of the US-made launchers, and has used them to attack Russian ammunition depots, command posts, and air defences.
Russia has repeatedly claimed to have destroyed a number of the systems.
Himars stands for High Mobility Artillery Rocket System.
Speaking to the Ukrainian-language arm of the US outlet Voice of America, Oleksii Reznikov said Ukraine still had all the systems it has been given.
The claim was later repeated on the official Twitter account of the country’s defence ministry.
.@oleksiireznikov for @holosameryky
🚮30% of russian elite troops (paratroopers, marines, special forces) were eliminated in Ukraine
✅#UAarmy has not lost any #HIMARS
🔄We need a political decision about ATACMS supplies
⏩Our plan: total de-occupation of 🇺🇦 territories
The seizure of the Zaporizhzhia nuclear power plant by Russia has raised the risk of a nuclear incident at the site, the Nato secretary-general, Jens Stoltenberg, has said.
The area around the plant, situated in the southern Zaporizhzhia region, has been under Russian control since March. The plant remains near the frontline and has been hit by shells multiple times in recent weeks.
Speaking at a joint press conference with the Serbian president, Aleksandar Vučić, Stoltenberg said: “The seizure of the Zaporizhzhia nuclear power plant by Russian forces poses a serious threat to the safety and the security of this facility, raises the risks of a nuclear accident or incident, and endangers the population of Ukraine, of neighbouring countries, and of the international community.
“It is urgent to allow the inspection by the International Atomic Energy Agency and to ensure the withdrawal of all Russian forces.”
Chinese troops will travel to Russia to take part in joint military exercises led by Russia, China’s defence ministry has said.
Last month, Moscow announced plans to hold “Vostok” [East] exercises from 30 August to 5 September, saying that some foreign forces would also participate.
China said the exercises would include forces from India, Belarus, Mongolia and Tajikistan.
The exercises will be the first conducted by Russia since 2018, when China took part for the first time.
The ministry said the exercises were “part of an ongoing bilateral annual cooperation agreement with Russia” and “unrelated to the current international and regional situation”.
Four more vessels carrying agricultural products have sailed from ports in Ukraine, according to the Turkish state-run news agency Anadolu.
Citing Turkey’s Ministry of National Defence, the agency said the ships departed the ports of Odesa and Chernomorsk on Wednesday morning.
They were carrying sunflower meal, sunflower oil and corn, it said.
On Tuesday, the first ship to depart Ukraine carrying wheat under the terms of a grain export deal agreed last month docked in Syria.
Since 1 August, a total of 21 ships carrying agricultural products have left Ukraine.
At least 12 Russians have been killed in a strike on a base in the occupied city of Nova Kakhovka, according to the Ukrainian military.
Footage posted to Telegram by the State Border Guard Service showed numerous burnt-out trucks, collapsed buildings, and debris.
“The base of the occupiers was destroyed in Nova Kakhovka,” the post read. “At least 12 Rashists [supporters of Russian militarism] were liquidated.”
A “freeze” of the conflict in Ukraine of the sort that followed the Russian occupation in Donbas is currently not possible, the Ukrainian defence minister has said.
Speaking to the Ukrainian-language arm of US outlet Voice of America, Oleksiy Reznikov said: “I don’t think that the option of freezing is possible.
“The option of reducing activity is possible, depending on the season. And again, modern war is a war of resources.
“Resources, of course, are being depleted on both sides. And, accordingly, each side needs time to recover.”
Reznikov also claimed that Russian forces had lost at least a third of their combat capacity since the start of the invasion in February.
Russia expects its average price for gas exports to more than double over the course of this year, according to economy ministry forecasts seen by Reuters.
The figures suggested that the average price charged by state-owned energy giant Gazprom in 2022 would be $730 (£605) per 1,000 cubic metres, up from $305 (£253) in 2021.
Based on those forecasts, the ministry expects Russia to earn $338bn (£280bn) and $256bn (£213bn) from its total energy exports this year and next, respectively, up from $244bn (£202bn) in 2021.
The forecast also said that, after this year, prices would gradually fall until the end of 2025.
The reduced supply of Russian gas to Europe since the invasion of Ukraine has seen prices surge. Gazprom has said supplies to China are increasing, but Europe remains by far the largest market for Russian gas.
Brad Zellar Used His Insomnia to Write His First Novel – Mpls.St.Paul Magazine
It took a lifetime of sleepless nights for Twin Cities writer Brad Zellar to finally wake up to the idea for his debut novel Till the Wheels Fall Off.
by Steve Marsh
August 20, 2022
11:56 PM
Photos by Roy Son
If Brad Zellar—photographed inside his magical writing shed in Merriam Park—looks familiar to you, it might be his strong resemblance to his brother Martin, front man for the Gear Daddies.
Brad Zellar didn’t really sleep last night. And it’s not because he’s so excited about the imminent debut of his first novel, Till the Wheels Fall Off, a coming-of-age novel about an insomniac searching for a father figure set in a lightly fictionalized small-town Minnesota roller rink in the 1980s. And it’s not entirely because it was so hot and sticky last night in his Dwell-worthy writing shed in the backyard of the Merriam Park house he shares with his wife, Kate. It’s because Zellar never really sleeps.
“I hate the heat because it fucks with my sleep even more,” he says. “Nights like last night bring back bad memories of shitty apartments and childhood insomnia.”
Because of this lingering condition, Zellar keeps “Hong Kong hours,” meaning after tossing and turning right through the dawn, he’s usually ready to hang by late afternoon. So, we’re sitting out in his writing shed, surrounded by industrial-strength shelves supporting his elegantly organized collection of vinyl records and art books and Alec Soth prints and vintage baseball mitts and watercolors of his dog.
At 60, there are some streaks of silver in Zellar’s salad, but he still favors the tees and jeans of his youth, like the yellow Dock Ellis T-shirt (the Pittsburgh Pirate who threw a no-hitter on LSD back in 1970) he’s pairing with his taupe Carhartts today. Last year’s excellent Pharoah Sanders/Floating Points album is softly spinning on the turntable, and it’s pleasantly warm in the shed, with just an oscillating fan to alleviate the height of a St. Paul midsummer afternoon. The space was designed, or at least expertly drawn up on the back of an envelope, by Michael McGuire, Zellar’s 93-year-old architect father-in-law to be cool in the summer and warm in the winter. And it’s working perfectly today—full of that comfortably moody, indirect, chiaroscuro-ready Rembrandt light. It’s the ideal creative space, full of art and literature and music that he’s spent his entire life restlessly searching for. But even this optimal environment hasn’t provided him with any, you know, rest.
Zellar has been afflicted with a severe sleep disorder for as long as he can remember, and his obsession with memory is one of the most potent side effects of his condition. His sleep doctors have told him he keeps lingering in the hypnagogic phase of sleep, what he’s come to understand as sleep’s foothills.
According to Zellar, the average person spends five to ten minutes climbing those foothills, hopefully settling into a pattern, their mind beginning to freely associate, not fully sleeping, not fully conscious, just starting to become untethered, and then their balloon goes off. Zellar’s balloon never goes off.
“So, I would just sit there all night—I still do—and all the shit would flow by: You hear voices, fragments from the day, snippets of songs, and dog tags jingling in the next room and cars going by. It’s like this stream of basically concrete music, like this endless turning of the radio dial, going up and down, all night.” He sighs. “That’s my brain at night. I can find a million metaphors for it because you can find them everywhere when you live with it.”
He’s tried everything—uppers, downers, screamers, and laughers, prescribed and self-prescribed, weaned off, and on, and off again—ever since he was diagnosed with attention deficit disorder as a teenager, growing up as one of two middle sons in his family of six in Austin, Minnesota.
“I’m not somebody who’s anxious,” he says. “The biggest curse of my combination of being tired all the time and having ADD—which I’m sure is the product of being tired all the time—is just not being able to focus. So, you’re bored all the time because you can’t really pay attention to anything.”
That might explain why it took him until 60 to drop his debut novel. Not that he hasn’t been busy. Before spending the better part of a decade making beautiful art books and indie newspapers with the great photographer Alec Soth, he spent a decade kicking out the jams on a popular blog for City Pages and The Rake (“Open All Night” at CP and “Yo, Ivanhoe!” at The Rake).
For years, Zellar has spent his Hong Kong office hours at his Brother manual typewriter (he can’t write on his computer because it’s connected to the insomniac’s worst nightmare: the internet), recording everything he can think of in a sort of sleep-deprived diary. He’ll record an annotated version of the day’s events, or lists of the books read or the records listened to.
“At some point, because I’m OCD, I decided that I’m going to do this thing every night,” he says. “It’s going to be like Scheherazade: I’m going to tell stories to keep myself alive.”
So, he wrote longhand, in a Moleskine notebook, and if it was any good, he would transfer it to loose-leaf via his typewriter.
“This was my way of paying attention and feeling like I was creating a record or a document,” he says. “Like I was somehow important, or my life was interesting, because I spent so much of my life feeling insignificant.”
Zellar is one of those writers who knew he wanted to be a writer from the age of 5. This was a somewhat audacious dream in Austin, Minnesota, in the 1970s. His father owned and ran a tire shop, Zellar Tire, in nearby Hollandale, and his mother was a talented, inventive, funny stay-at-home mom, quick with an impression or an imagined-on-the-spot character.
“She was constantly casting us in one-act plays,” he says.
Things were great until a string of health problems put his father in a Rochester hospital for 269 straight days—it was the first of seven open-heart surgeries. “He was basically in and out of hospitals from the age of 30 until his death at 69,” he says.
The trauma of his father’s long illness deprived him of his playful and creative mom, and maybe something stalled out in Zellar, as well. He was always a voracious reader, but somehow his dream of becoming a writer kept getting deferred: first by a rebellious incident that got him banned from the school newspaper, and then by his own decision to blow off college, to just move up to Minneapolis and begin working a series of self-described “shit jobs.” He worked the third shift at Control Data on Washington Avenue, he worked in a warehouse loading shipping containers, and he worked as an overnight parking lot attendant. “I was resigned to being a grunt for the rest of my life,” he says.
Zellar always wondered why people in the novels he read never worked a real job—they were always doctors or lawyers or some other manifestation of their plot utility—so maybe this string of menial jobs was Zellar’s self-flagellating way of keeping it real. Regardless, he realized that keeping it real totally sucked, but maybe he could make it entertaining, so he began publishing a zine called Scread.
“You know,” he says, as he notices my blank expression. “Like screed plus read?”
The zine got noticed by an editor at City Pages, and all of a sudden, in his early 30s, Zellar was offered the first in a parade of writing jobs that he never applied for.
He used this opportunity and every one that followed to pursue a beat that he understood all too well: obsessing over the overlooked, but in no way insignificant, figures living out their lives of quiet desperation, or vibrant eccentricity, oftentimes in the wee hours of the morning. For years, he interviewed subjects like the denizens of an all-night Ping-Pong parlor on Lake Street, or the retired Shriners dining at the Red Fez Grill in the Shriners mansion by the Swedish Institute, or a well-respected proprietor of a cavernous magic shop in the shadow of the Metrodome. He tracked down countless derelicts, outsiders, and weirdos, people that had the most unlikely but interesting stories to tell. He had a knack for finding these figures and capturing the hookiest aspects of their spirit, probably because he was a fellow traveler.
And then, just as the world of alternative weeklies was becoming another dour corporate media relic, it was this knack that helped Zellar land the most magical, serendipitous assignment of them all, a freelance writing gig with Mr. Sleeping by the Mississippi himself, one of the great image hunters of weirdo Americans that America has ever seen: Alec Soth.
Soth took Zellar on the most unlikely ride of his life: traveling around to seven different regions of the United States, each time returning home to put together a special-edition newspaper called The LBM Dispatch.
“There’s a William Maxwell quote that I love that’s motivated me my whole life,” he says. “All these people who spend their lives looking for the hole in the net but never find it. I was always like, ‘I’m going to find the hole in the net.’ And when I met Alec, he was looking for the hole in the net. And then we realized we were both looking for the same thing.”
Over six years, they produced some of the most poetically idiosyncratic collections of images and text ever created: seven editions of The LBM Dispatch, a chapbook called Conductors of the Moving World, and an illustrated “method-acted” picture novel by the wholly invented author Lester Morrison entitled House of Coates. That’s not to mention the time he was the subject of Soth’s set-in-the-desert fashion shoot for Russian Esquire.
“Somebody said, ‘Be careful, those are $2,000 shoes,’” he says, “and I remember thinking, Who buys $2,000 shoes?”
In 2015, their partnership culminated with one last Dispatch in conjunction with The New York Times Magazine and a joint visiting-artist residency at the University of Wisconsin–Madison. They both sort of simultaneously realized that their too-good-to-be-true partnership couldn’t last forever. Soth was better off making photographs he could sell through his gallery, and Zellar met his future wife, Kate, at a House of Coates book launch, married her, moved to St. Paul, and was finally forced to think about what he was going to do with 28 years’ worth of nocturnal record keeping. He had been exercising this spiritual practice for so long, and had accumulated so much material, that he knew he had a book in him. “I had seven or eight book-length projects in here,” he says. He realized his biggest problem, to bastardize a Lewis Carroll idea, was deciding: Which book did he want to write?
Photo by Caitlin Abrams
Till the Wheels Fall Off, Zellar’s debut novel, is out now from Coffee House Press.
Zellar says he’s always been fascinated by guys who wrote memoirs that were busted for being a pack of lies—so what if he wrote a novel that was essentially true? So that’s what he’s given us with Till the Wheels Fall Off. It is a product of his sui generis production cycle: Moleskine to typewriter to loose-leaf, then scanned in by an enterprising husband-wife team of agents and given a prodigious line edit conducted by some genius at Knopf, even though it went on to be published by Coffee House Press. And it concerns an area of the country—southern Minnesota—that in his estimation has been ignored by the greats. And it’s about a small-town kid with a sleeping disorder who never knew a father who was killed in Vietnam, ending up with a traumatized mother before lucking into a stepfather who was the proprietor of a magical roller rink with an insane record collection.
Basically, it’s all based on a true story: Zellar’s life, just poetically novelized through his dusky, neurodivergent process. It almost feels counterintuitive that the book is so imbued with a quixotic, fanciful, straight-up Capra-esque It’s a Wonderful Life quality. But Zellar says this was the only truly honest move. He’s honest with himself about how good he’s had it, about how even his biggest curse has helped him become the writer he is. Zellar realizes that even without the ability to approach the dreamwork the way the rest of us have, his dreams have come true.
Steve Marsh is a senior writer at Mpls.St.Paul Magazine.
August 20, 2022
11:56 PM
Sign up for our daily newsletter.
The arts and entertainment blog. Read more
© Key Enterprises LLC
All rights reserved
Key Enterprise LLC is committed to ensuring digital accessibility for mspmag.com for people with disabilities. We are continually improving the user experience for everyone, and applying the relevant accessibility standards.
NEW JUROR ANNOUNCEMENTS FOR 2022 COMPETITIONS – austinfilmfestival.com
In addition to an 8 day film festival and 4 day Conference, we offer Year Round Events, a Young Filmmakers Program, a TV show, Radio show, Podcast and more!
Austin Film Festival is really about creativity, about trying to help people find their voice. And it’s fun.”
– Ron Howard, 2009 Extraordinary Contribution to Filmmaking Award Recipient
– Ron Howard, 2009 Extraordinary Contribution to Filmmaking Award Recipient
The folks [at AFF] share our passion…Which is, of course, why they are so damn adept at crafting a conference that manages to satisfy the needs and desires of all of us attending.”
-Scott Rosenberg, screenwriter
-Scott Rosenberg, screenwriter
Get to know the faces and stories of the people who run the show for the Austin Film Festival!
Want to work with us? We’re always looking for folks to join our team whether it’s through volunteering or interning or working with us on staff. Check out our current opportunities below.
– David McKenna (BLOW, AMERICAN HISTORY X)
Shelli Wright, 2015 Finalist
This competition accepts both filmed and written submissions in an effort to find talented voices who can adapt their vision to emerging digital platforms. AFF will select one winner who displays the strongest voice and greatest potential for establishing a digital series.
At AFF, we love being able to further the art and craft of storytelling in the next generation! The Young Filmmakers Competition is a FREE film competition is open to young creatives ages 13-18.
AFF Blog, Announcements, Competitions, Film Competition, Juror Announcements, News, Screenplay Competition
Take a look at the 2022 Jurors for the Film and Script Competitions, below!
Devon Greggory
Drama & Comedy Spec
Born the youngest of nine, in the heart of Dixie, Devon Greggory grew up escaping the harsh reality of the projects through film and television. Years later, this Alabama native would pack his bags and run away to pursue his dream of filmmaking. Homeless, hungry and living on the streets of LA, he failed. But in failure, he found resilience. Devon would refocus, obtain his JD from Emory University School of Law and join Jones Day, one of the world’s foremost law firms, as a star associate. But, after a season of legal practice, Devon left the comfort of law to again chase that dream; this time by earning his MFA in Production from the University of Southern California where he would hone his craft, winning the Jeffrey Jones Writing Award and becoming a finalist for both the Sundance Filmmaker’s Lab and Screenwriter’s Lab.
As a director, over 35 film festivals have recognized his work, including 12 finalist and 4 “best of” selections. Devon has also helmed several commercials, an award-winning music video and a critically lauded stand-up comedy feature. But, it was as a writer where Devon truly found his passion… he’s penned two produced features (Sacred and Hair Show), written several pilots including, Black Wall Street, an event series about the Tulsa Race Riots, for HBO Max; Marlowe, a music-driven PI drama executive produced by Aaron Kaplan for The CW; and a reboot of the iconic series, The Game, which Devon executive produces for Paramount+.
His credits also include stints on WGN’s critically acclaimed drama, Underground; CBS’s franchise spin-off, CSI: Cyber; and BET’s ground-breaking hit, Being Mary Jane… and he was the creative force behind BET’s Soul Train-inspired period drama, American Soul, as Co-Creator, Showrunner and Executive Producer.
In success, Devon founded The Inphiniti Company, a full-service production house where he serves as President and CEO. Through Inphiniti, Devon is focused on creating content that touches the heart, inspires the soul, challenges the mind and compels you to action. Currently in a multi-year overall deal with CBS TV Studios, Devon, through Inphiniti, will be writing, directing and producing content for CBS, Paramount+, The CW and various ViacomCBS-owned platforms. Grateful for his success, Devon refuses to celebrate until his entire dream is reality. As he puts it, “The only success that interests me is fulfilling my God-given destiny, caring for my family, and becoming a better man. If film and TV is the conduit through which this is accomplished then so be it; if not, no matter, it’ll all be accomplished.”
Devon is repped by CAA, manager Steve Simons of The Shuman Company and attorney John Meigs of Hansen, Jacobson, Teller, & Hoberman. He serves as an adjunct professor at his alma mater, USC. And, he currently resides in Los Angeles with his beautiful wife and two amazing sons.
Brian Siegele
Fiction Podcast
Brian Siegele is a Colorado and Los Angeles-based film, television, and audio creator. A graduate of the USC School of Cinematic Arts, he is a former Austin Film Festival Screenplay Winner (2013) and Podcast Finalist (2019). Though he began his career writing tentpole features, he has since pivoted to generating high-concept IP for the podcast ecosystem. His first audio series BLACK BOX starred Joel McHale and Kelsey Grammer and being set up for television, and he is now at work on several new series.
Jameel Saleem
Comedy Pilot
Jameel began his professional writing career with the 2012 nationwide release of his indie film, EXIT STRATEGY. After producing three indie feature films and a web series, Jameel set his sights on television. He wrote and produced a pilot for NBC Playground and soon after staffed on his first network show, ABC’S MANHATTAN LOVE STORY in 2014. Since then, he’s worked on multiple TV projects and films. He currently serves as a Co-Executive Producer and Writer on BOB’S BURGERS, Writer and Producer for Netflix’s DISENCHANTMENT and Writer for SOUTH PARK DIGITAL – and has several TV and film projects in development, including an animated film with his mentors South Park’s Matt Stone and Trey Parker.
Kelly Edwards
Drama Pilot
Author The Executive Chair; Writer; Recovering Development Executive. Former executive and current producer, tv writer, and book author Kelly Edwards has developed shows like Living Single, Girlfriends, The Parkers, Clueless, Two Guys and a Girl, as well as a tv drama series, a Hallmark movie, and award-winning shorts for HBO.
For seven years, she served as the head of Talent Development for HBO, launching the careers of emerging artists before herself becoming a Sundance Episodic Lab fellow, releasing her Amazon bestselling book The Executive Chair: A Writer’s Guide To TV Series Development, and staffing on Our Kind of People.
Edwards holds a B.A. in Theater from Vassar and an MFA in Writing for TV and Film from Emerson College. She has been a member of the Annenberg Inclusion Board, the ATAS Diversity Committee, the United Negro College Fund Leadership Council, and a NALIP trustee. She has also served as President of her non-profit, Colour Entertainment, which has provided mentorship and leadership opportunities to creative executives for over twenty years.
VJ Boyd
Drama Pilot
VJ Boyd most recently worked on the upcoming JUSTIFIED spinoff, CITY PRIMEVAL, for FX/Hulu. He previously wrote for SWAT and JUSTIFIED and co-created LINCOLN RHYME: HUNT FOR THE BONE COLLECTOR. He is also the co-author of the IDW comic book series NIGHT MOVES.
Marc Bernardin
Sci-Fi Feature
Marc Bernardin is a WGA Award-winning television writer-producer who has worked on Star Trek: Picard, The Continental, Carnival Row, Treadstone, Castle Rock, Critical Role: The Legend of Vox Machina, Masters of the Universe: Revelations, and Alphas. In an earlier life, he was a journalist for the Los Angeles Times, The Hollywood Reporter, Playboy, and Entertainment Weekly. In comics, he’s an Inkpot-winning writer of Adora and the Distance, Peter Parker: The Amazing Shutterbug, Genius, The Highwaymen, and Monster Attack Network. And he cohosts the Fatman Beyond pop-culture podcast with Kevin Smith.
Nicole Perlman
Sci-Fi Feature
Nicole Perlman is best known for her work on the Marvel Studios’ blockbusters, Guardians of the Galaxy and Captain Marvel and on Legendary Studios’ Detective Pikachu. Nicole is a Cinereach and Film Independent Directing Fellow, and has twice been awarded the Tribeca Film Festival’s Sloan Foundation Grant for Science in Film, along with the Hugo Award, and the Ray Bradbury award for Outstanding Dramatic Presentation. Nicole is currently developing a supernatural horror series for Apple+ TV and is casting her first feature to direct.
Oren Uziel
Comedy Feature
Oren Uziel’s writing credits include THE LOST CITY, 22 JUMP STREET, MORTAL KOMBAT, THE CLOVERFIELD PARADOX, ESCAPE ROOM: TOURNAMENT OF CHAMPIONS, FREAKS OF NATURE and SHIMMER LAKE, which he wrote and directed. He’s currently working on a number of projects, including the sequel to SPENSER: CONFIDENTIAL, starring Mark Wahlberg, and CLUE! starring Ryan Reynolds. Oren is also finally dipping his toe into TV with an adaptation of the 1946 film noir, THE STRANGE LOVE OF MARTHA IVERS.
Stephanie Allain
Comedy Feature
Stephanie Allain’s Homegrown Pictures is a film and television production company dedicated to creating content by and about women and people of color with authentic stories, depictions and representation. Allain, who made history in 2020 when she became the first Black woman to produce the Academy Awards (for which she was Emmy-nominated), has produced Hustle & Flow, Something New, Peeples, Beyond The Lights, Dear White People, French Dirty, Burning Sands, Juanita and Really Love. Her films have been nominated for numerous awards with wins from Academy of Motion Pictures Arts and Sciences, Film Independent Spirit Awards, NAACP and others. In television she executive produced four seasons of Justin Simien’s Netflix series, Dear White People, and is currently in production on her first documentary Untitled Billy Preston. Her first book, I’M IN CHARGE! was acquired in an auction by Candlewick Press. The picture book, slated to be published in Spring 2024, models for children how they can take charge of their bodies, voices and choices. Gabrielle Ebron is Director of Development for film and TV. Evadne Eddins serves as Creative Executive.
Chuck Hayward
DIGITAL SERIES CATEGORY JUROR
Emmy® nominated screenwriter and producer Chuck Hayward has spent the last twenty years building an impressive career. He has amassed an enviable list of credits in television and film, working with some of the most well-respected names in entertainment. Upcoming, Hayward was brought on as a Co-Executive Producer to both the Emmy® award winning comedy “Ted Lasso” for the show’s highly anticipated third season, and to second season of Hulu’s “Life And Beth.” Most recently, Hayward has garnered nominations from the WGA, Critics Choice, and the PGA respectively for his work on “WandaVision.” In 2021, he received two (2) Emmy nominations for his acclaimed work on “WandaVision,” which is unanimously hailed as a critical achievement for the Marvel Cinematic Universe. The genre-bending series shines a heartfelt spotlight on two of the MCU’s most intriguing characters as it weaves together a mysterious and mesmerizing story. The original live-action series debuted exclusively on Disney+ in January 2021. Additonally, Hayward is the screenwriter and an executive producer of the remake of the iconic 90’s cult classic DON’T TELL MOM THE BABYSITTER’S DEAD, which begins production in 2022. His previously produced feature projects were the highly rated the Netflix comedy feature STEP SISTERS, and the indie film FAT CAMP. Other notable credits include Netflix’s critically acclaimed series “Dear White People,” ABC’s “Mixed-Ish,” and Showtime’s “Flatbush Misdemeanors.” While working on this wide variety of shows, Hayward honed his skills as a producer as well as a writer. Prior to this, he was staffed on several beloved, yet short-lived sitcoms, including “Bent” and “One Big Happy” on NBC, “Cooper Barrett’s Guide to Surviving Life” on FOX, and Nickelodeon’s “Wendall & Vinnie.” In the earlier part of his career, Hayward was accepted into multiple programs for up and coming writers, including but limited, to 20th Centry Fox Television’s FOX Writers Intensive, the WGA’s Feature Writer Access Project as well as the WGA’s TV Writer Access Project. A distinguished alumni of Syracuse University’s S.I. Newhouse School of Public Communications, Hayward began his career as a producer’s assistant at Spirit Dance Entertainment and State Street Pictures, then transitioned to working as a production assistant on beloved television series such as “Everybody Hates Chris” and “Entourage.” When he’s not working, Hayward donates his time and resources to local charities such as “My Friend’s Place, and “Kids In The Spotlight.” Additionally he lends his voice and vast industry knowledge while serving as a juror and consultant for the annually held Nashville Film Festival and the Austin Film Festival. Hayward credits his career success to supportive mentors, friends, and family with a focus on “paying it forward” for up-and-coming creatives. He resides in Los Angeles, where he enjoys anything related to the beach, in addition to “cobbling together some semblance of a social life.”
Amy Talkington
DIGITAL SERIES CATEGORY JUROR
Amy Talkington is an award-winning writer, director and producer living in Los Angeles. In the feature arena, Amy most recently wrote “Valley Girl” (MGM’s musical remake of the 80s cult film) and has written numerous other studio features that are in various phases of development, including “Undercover” at Lionsgate with Steve Pink directing. In the television arena, Amy was a Co-Executive Producer and writer on Hulu’s hit series “Little Fires Everywhere” (starring Reese Witherspoon and Kerry Washington) for which she was nominated for both an Emmy and a Writers Guild Award. Amy currently has several series in development including pilots at FX, 20th Television, ABC Signature and Film Nation. Previously, Amy wrote and directed the independent feature film “The Night of the White Pants” as well as five short films which won multiple awards and played festivals all over the world, including Sundance and The Austin Film Festival. For more info, check out www.amytalkington.com.
Roy Lee
DARK MATTERS FEATURE CATEGORY JUROR
Roy Lee is a film and television producer and founder of Vertigo Entertainment. Lee has an undergraduate degree from GWU and attended law school at American University. His first producer credit was for THE RING (2002), and from there he went on to produce many of the iconic American horror films of the past two decades including THE GRUDGE, THE STRANGERS, and IT. Lee also produced the beloved children’s animated films THE LEGO MOVIE, and HOW TO TRAIN YOUR DRAGON. Currently, Roy is producing a diverse slate of popular video game adaptations, acclaimed-director-driven thrillers, television series at many of the major streaming services, and more horror films.
Jessica Mendez Siquerios
NARRATIVE SHORTS FEATURE CATEGORY JUROR
Jessica Mendez Siqueiros is a Chicana writer/director of mixed indigenous Sonoran and European ancestry seeking to normalize complex & authentic narratives about the Southwest Mexican-American community through film. Her short films have screened at over 75 festivals, including Palm Springs, Seattle, Cleveland, Hamptons, and the Los Angeles Latino Film Festival. They have garnered an Audience Pick from Palm Springs ShortFest, the KidsEye Grand Prize for Best Live Action Short from Flickers Rhode Island Film Festival, Best Narrative Short (Comedy) from Cinequest, among other awards. Jessica’s short film, The Last of the Chupacabras, was produced by Walt Disney Studios as part of the inaugural class of their Launchpad Shorts Incubator, and is currently streaming on Disney+. Her upcoming short film Machisma is being made through the support of XRM Media & The Sundance Institute. She was the recipient of the 2021 Latinx Fellowship from the Sundance Feature Film Program.
Orian Williams
NARRATIVE FEATURE CATEGORY JUROR
Orian Williams Born in Jackson, Mississippi, raised in Houston, Texas, and Orian graduated from Baylor University and came to fame with the Academy Award nominated film SHADOW OF THE VAMPIRE, which he produced with Nicholas Cage. Orian continued his success producing the BAFTA and Cannes-winning CONTROL, directed by Anton Corbijn, and BIG SUR, which opened at the Sundance Film Festival in 2013. Other notable films include ENGLAND IS MINE directed by Academy Award-nominated writer/director Mark Gill; CURTIZ about the making of CASABLANCA; and CREATION STORIES, which Orian produced alongside Danny Boyle. Most recently, Orian worked with Amy Adams to produce SAM & KATE starring Dustin Hoffman and Sissy Spacek, and is currently directing and producing a biopic about legendary singer Jeff Buckley starring Reeve Carney. In 2007 Orian was chosen by Variety Magazine as one of Hollywood’s top producers.
Hannah Fidell
NARRATIVE FEATURE CATEGORY JUROR
Hannah Fidell is a writer, director and producer of television and film who is best known for creating the FX show ‘A Teacher’ which was based on her movie of the same name.
Matt Lieberman
COMEDY VANGUARD FEATURE CATEGORY JUROR
In a span of about 2 years, Matt Lieberman has had seven screenplays produced by major studios. These include his Black List Free Guy script due August 13, 2021 (starring Ryan Reynolds with Shawn Levy directing), Rumble due to release February 18, 2022 (starring Will Arnett), Scoob! released on May 15, 2020 (starring Will Forte, Zac Efron and Mark Wahlberg), Playing with Fire released November 2019 (starring John Cena), and The Addams Family (which grossed over $100 million dollars domestically in 2019). In addition, he wrote The Christmas Chronicles (2019’s Netflix holiday hit from his spec script starring Kurt Russell) and its sequel which was released November 25, 2020, directed by Chris Columbus. Most recently, Lieberman set deals for two big projects. Netflix preemptively acquired Walter the Farting Dog, for Matt to adapt, based on the beloved children’s book. He also sold his high concept pitch entitled Life Hacks to Sony Pictures with Chris Hemsworth attached to star and produce. Other projects in development for Lieberman include his spec script Yumanzu for Paramount and another spec, Meebo and Me for Warner Bros., both of which he is producing alongside Adam Kolbrenner and Kenya Barris. Originally from New Jersey, Matt is a graduate of NYU’s Tisch School of the Arts. He lives in Los Angeles with his wife, daughter and French bulldog.
Lisa Freberg
ANIMATED SHORTS CATEGORY JUROR
Producer / Executive Producer Lisa J Freberg, is the Co-founder & Managing Partner of One Good Man Productions based in Los Angeles, CA. She is a former Dreamworks Animation executive having overseen development and production for branded content for franchises including How To Train Your Dragon, Guardians, Kung Fu Panda, Shrek Franchise, and Trolls. She produced Penguins of Madagascar Got Your 6 featuring FLOTUS Michelle Obama, Book of Dragons created and directed by veteran animation storyteller, Steve Hickner. She’s had the honor of working with legendary actors, Dustin Hoffman, Bryan Cranston, Craig Ferguson, Jim Parsons, Rihanna, Jennifer Lopez to name a few. She is the recipient of Women-In-Film Finishing Fund for her feature documentary, The Scarlet Bond. Lisa produced multiple award-winning campaigns for Nike, Budweiser, Pepsi, Coca Cola, Verizon. She was the Producer of the landmark 3D Pepsi Sobe Super Bowl Ad combining 3D live action and animation from multiple studios. She’s produced many VFX, animation & live action projects. In 2019, One Good Man Productions was tapped to lead the creative development and production of six digital series including, Adulting with Richard Sherman and pop culture phenomenon DJ, Marshmello’s Power video and multiple docs-follow series. Her One Good Man Productions team simultaneously delivered all six shows within the same year, an impressive accomplishment for an independent production company. As if that was not enough to keep her busy, on the tail end of this work, she produced producing feature film, Alex / October and Three Headed Beast both completed in 2021-22. Alex / October starring award winning actor, Brad Hunt, Tara Shayne and Ethan Cutkosky hits theaters and On Demand fall 2022 with distribution through Deskpop Entertainment spearhead by Mat Levy. Selected as one of twelve films in narrative competition at the prestigious Tribeca Film Festival 2022, the critically acclaimed, Three Headed Beast made it’s world premiere as the debut feature film of Austin filmmakers writer / director team, Fernando Andres & Tyler Rugh. Three Headed Beast continues on the film festival circuit through 2022-23. Her television creates include her work as creator & Executive Producer of GloZell Is you Okay? original lifestyle series scheduled to premiere on Crackle Plus 2023. She is currently working on the adaptation critically acclaimed award-winning novel UnBroke Horses by novelist D.B. Jackson as a limited series and developing her next feature film, VINCENT with award winning writer /director, Deran Sarafian.
Stephen Christy
DARK MATTERS FEATURE CATEGORY JUROR
Stephen Christy is the President of Development for BOOM! Studios, one of the largest comic book and graphic novel publishers in North America. He currently manages BOOM!’s film and TV development slate and supervises BOOM!’s first look feature and TV deals with Netflix. He served as Producer on the 20th Century Fox supernatural thriller The Empty Man, was an Executive Producer on the Disney+ TV series Just Beyond, and is currently Executive Producing an animated TV adaptation of Mech Cadets for Netflix and an animated TV adaptation of Hex Vet for Nickelodeon. In his previous life as Editor-in-Chief for comic book publisher Archaia Entertainment he became the first editor in the industry to win back-to-back Eisner Awards for Best Original Graphic Album two years in a row, for Jim Henson’s Tale of Sand and Return of the Dapper Men. He currently lives in Los Angeles and is a graduate of Emerson College.
Jameel Saleem
ANIMATED SHORTS CATEGORY JUROR
Jameel Saleem is an Emmy-nominated writer and producer working in film and television. He currently serves as a Co-Executive Producer and Writer on BOB’S BURGERS, Writer on Netflix’s DISENCHANTMENT, Writer for SOUTH PARK DIGITAL and has several TV and Film projects in development.
Neil Miller
TEXAS SHORTS CATEGORY JUROR
Neil Miller is the founder and publisher of Film School Rejects, an Austin-based online magazine dedicated to quality criticism and helping readers understand the filmmaking process. He is also an Executive Producer of “One Perfect Shot” on HBO Max, a docuseries about filmmaking inspired by the One Perfect Shot Twitter account, which he owns and operates. As well as the co-host of “Trial By Content,” a pop culture debate podcast on The Ringer Podcast Network, and the independent podcast “Da7e and Neil’s Pop Culture Adventure.”
AFF Blog, Announcements, Competitions, Film Competition, Juror Announcements, News, Screenplay Competition
©2022 Austin Film Festival • Austin Film Festival is a Not for Profit 501(c)(3) Corporation.
Site by:BeiBay• Custom icons & base colors by Mosak.Sponsors.Privacy Policy.Store & Refund Policy.AFF Policies.Contact.Sitemap.On Story
We value your privacy and want to be clear about the data we collect, how we use it and your rights to control that information. Considering recent changes established by the General Data Protection Regulation (GDPR), a set of laws passed in the European Union, we have updated our Privacy Policy.
We will not sell, distribute or lease your personal information to third parties unless we have your permission or are required by law to do so. We may use your personal information to send you promotional information which we think you may find interesting if you tell us that you wish this to happen.
Privacy Policy
Like many services, Google Analytics uses first-party cookies to track visitor interactions as in our case, where they are used to collect information about how visitors use our site. We then use the information to compile reports and to help us improve our site.
Cookies contain information that is transferred to your computer’s hard drive. These cookies are used to store information, such as the time that the current visit occurred, whether the visitor has been to the site before and what site referred the visitor to the web page.
Overall, cookies help us provide you with a better website by enabling us to monitor which pages you find useful and which you do not. A cookie in no way gives us access to your computer or any information about you, other than the data you choose to share with us.
Google Analytics collects information anonymously. It reports website trends without identifying individual visitors.
Cookies Used
https://tools.google.com/dlpage/gaoptout
Blue Vista Acquires Residential Property Located Near Purdue University – Business Wire
CHICAGO–(BUSINESS WIRE)–Blue Vista Capital Management (“Blue Vista”) today announced it has acquired Wabash Landing Apartments, a 206-unit residential property located near Purdue University in West Lafayette, Indiana. The Property serves a diverse population of students and faculty at Purdue University, as well as local professionals. The transaction is part of Blue Vista’s ongoing strategy of providing institutional investors with access to unique and differentiated core plus residential real estate opportunities with long-term hold strategies. Blue Vista’s affiliate student housing management company PeakMade Real Estate is managing the property.
Wabash Landing Apartments is strategically located along the Wabash River proximate to Purdue University, within walking distance to restaurants, retail businesses, services and local transportation in the downtown West Lafayette area. The property consists of studio, one- and two-bedroom units, and offers residents a comprehensive amenity set including a rooftop deck, fitness center, resident lounge, courtyard and study room. The apartments offer 9-foot ceilings, walk-in closets, stainless steel appliances and in-unit washers and dryers. As of the end of July 2022, the property was 100% occupied.
“Wabash Landing is an example of Blue Vista’s strategic approach to executing a core-plus strategy across alternative residential property types, and commitment to delivering an exceptional rental experience,” said David Katz, Managing Principal and Fund Manager for Blue Vista’s middle market equity real estate funds. “Home to the highly acclaimed Purdue University, West Lafayette is one of the most active residential and student housing markets in the U.S., and is an opportune location for high-quality, purpose-built rental accommodation. We look forward to continuing our partnership with PeakMade to continue improving the attractiveness and efficiency of the Wabash Landing property.”
“Blue Vista’s investment in Wabash Landing reflects our commitment to developing top-tier, highly-amenitized residential experiences in high-growth areas for students and professionals alike,” said Jason Schwartz, Managing Principal and Co-Head of Real Estate Equity at Blue Vista. “Purdue University is experiencing rapid growth and increased enrollment numbers, and as of Fall 2021, welcomed its largest freshman class ever, with more than 10,000 students – representing total enrollment growth of over 8% from the prior year. We are pleased to deliver a high-quality residence option to students as demand in West Lafayette continues to rise.”
ABOUT BLUE VISTA
Blue Vista is a leading investment management firm focused on helping investors maximize returns through best-in-class real estate equity and credit strategies across the risk-return spectrum. Blue Vista offers investors the benefits of a vertically integrated platform, complementary strategies managed by experienced teams, and deep expertise as an owner, operator and lender across all property types and various markets in the U.S. and Canada. Since its inception in 2002, the firm has invested over $12.0 billion in total capitalization, with the goal of challenging the status quo, setting high standards and bringing a relationship-based, client-focused approach to real estate investing. For more information, please visit https://bluevistallc.com/.
PRESS
Carissa Felger / Alex Jeffrey
Gasthalter & Co.
(212) 257-4170
[email protected]
PRESS
Carissa Felger / Alex Jeffrey
Gasthalter & Co.
(212) 257-4170
[email protected]
Transcription service Otter is making free users pay to access older recordings – The Verge
Filed under:
Free users will have to pay to access more than 25 recordings
If you buy something from a Verge link, Vox Media may earn a commission. See our ethics statement.
Automated transcription service Otter is making some big changes to its offerings for both free and paying customers. Mostly, the company is downgrading its features — reducing the number of audio imports users can make; the length of audio they can transcribe each month, and so on — though it is giving free users access to some new tools.
One of the biggest changes, though, is that free users will no longer be able to access their full back catalogue of recordings. Instead, they’ll only have access to the most recent 25. The rest will be “archived” — that is, they’ll still exist on Otter’s servers, but users will have to either delete other conversations to access them, or pay to upgrade to Otter’s “pro” plan.
This and other changes to the service will kick in on September 27th, so any free users with more than 25 recordings may want to download their back catalogue before then. After September 27th, free users will still be able to access these recordings (by downloading then deleting audio files one at a time) but it’ll be more of a hassle.
You can see the full range of changes in the image above, and Otter also has a useful FAQ on what’s different. The general upshot, though, seems to be that the company has been too generous to users, and now needs to nudge more people onto its paid plan (which: fair enough! They’ve got to make money somehow).
It’s not all bad news, though: Otter is giving some new tricks to free users too, including access to an auto-join feature for meetings and free AI-generated summaries of recordings. These features were first added for paid users earlier this year, as part of Otter’s plan to position itself not just as a transcription tool, but as a multi-purpose work hub.
Subscribe to get the best Verge-approved tech deals of the week.
Please confirm your subscription to Verge Deals via the verification email we just sent you.
Oops. Something went wrong. Please enter a valid email and try again.
Take Five with Reva K – Guitar Girl Mag
As a shy girl growing up in Chicago, Reva K didn’t connect as well with others as she did with her first guitar. After playing and writing for over a decade, she studied music production under the mentorship of Grammy-nominated producer ill Factor for a year.
Her subsequent debut single “Color on the Walls” garnered critical acclaim from music-industry veterans and judges of an international songwriting competition. That’s when former manager of Taylor Swift connected Reva K with her current producer who she began working with on her third single “State of Mind,” which resulted in new media attention, a sharp increase in streaming numbers, and international FM-radio airplay.
Tell us a little about your musical background; how old were you when you started playing guitar, and what inspired your musical desire?
I don’t recall the exact thought process that led me to pick out a toy guitar for myself at age six but I know that I was confident about that decision. I got my first acoustic guitar at age eight and then my first electric guitar about four years later. That is the point when everyone around me realized that I was quite serious about pursuing music. I began to apply my guitar skills to more focused songwriting efforts and it was a regular part of my daily routine. As a shy and socially-awkward kid, songwriting felt like the most natural form of communication for me and perhaps that remains true to this day.
Where do you find the inspiration for your music?
Raw inspiration comes to me in many forms and often in between recording sessions. So, I have become pretty efficient at documenting moments of inspiration on my phone, which I can refer back to in the studio. They range from personal topics I can’t stop thinking about or specific phrases to basic chord progressions and top-line melodies. My creativity is consciously or subconsciously influenced by all the music I’m currently consuming. Right now, that includes Haim, Kacey Musgraves, New Order, Air, Lush and Eminem.
What is your songwriting process?
I constantly observe and process the world around me as well as my own reactions to it. When I’m unable to contain my thoughts and feelings, I set up a few mics, pick up a guitar and record whatever comes out. Sometimes I prefer to first program drum patterns and synth parts that I can riff over. I may compose ideas for many consecutive days before sitting down with my producer to discuss which demos should be developed further.
What’s your go-to guitar for songwriting?
For over 15 years, I primarily composed ideas using one of my acoustic guitars. When I started writing a new batch of songs in late 2021, I went to my semi-vintage Les Paul Studio and it remains my go-to guitar for both writing and recording.
Music to me is…
Music is undoubtedly my favorite art-form. It seems that music is woven into so many important aspects of my life. Beyond the studies related to my own musical expressions, I often find comfort in it and some songs become permanently associated in my mind with specific people, places, and moments in time.
Connect with Reva K
Instagram | Facebook | Spotify | Website
#Elon26: Class of 2026 formally welcomed to Elon during New Student Convocation – Today at Elon
By Trajan Warren, staff
August 20, 2022
More than 1,700 members of the Class of 2026 and 100 transfer students gathered with their families and friends Under the Oaks on Saturday to participate in this university tradition.
Shaded under the iconic oaks of Elon’s campus, the Class of 2026 and their loved ones were formally welcomed into the Elon community during the New Student Convocation ceremony on Saturday morning.
“Class of 2026, today commences your educational journey,” President Connie Ledoux Book said during the 2022 New Student Convocation.
“We are very protective of our community at Elon and it’s not something that we take for granted. We are deliberate and outspoken about the community values that allow everyone here to feel our welcome, a sense of belonging and safety. We know it is fundamental to creating an environment where you can learn and find a sense of purpose.”
The convocation began with a procession of the incoming students through the Alamance Building. Students passed the bell that hung in Old Main when the building along with others on Elon’s campus were destroyed by fire in 1923. In the nearly 100 years since the fire, the university has risen from the ashes to see tremendous growth and become one of the nation’s premier higher education institutions. The bell, which sits in the rotunda of Alamance, serves as an example of resilience.
“So, when you encounter a challenge at Elon, which you most certainly will, I hope you will come by the Alamance Building and touch that bell to recall the bright future we celebrate here today, and as a reminder to you of what it means to rise again, like a phoenix,” Vice President for Student Life Jon Dooley said.
Nadine Jose ’23, executive president of the Student Government Association, welcomed the Class of 2026 by saying that Elon would be a “liminal space” in their lives. “Liminal” derives from the Latin word “threshold.” Jose said Elon will be a liminal space in each student’s life, a time between “what was” and “what will be.”
But Jose had no reservations about the bright futures ahead of every student in the audience.
“Standing here today, I see a group capable of achievements beyond my wildest imagination. Whose collective abilities and experiences are diverse and rich, whose potential is palpable,” Jose said. “In this space, you are on the verge of something new. You found something here that ignited a fire within you, that felt right, that felt like home. That feeling inside of you is true, Elon is special.”
The overwhelming nature of Move-in Day, paired with the plethora of events and activities, had Ryan Campbell ’26 feeling slightly uneasy about the college experience. But the welcoming words and faces he encountered at New Student Convocation and advice that it’s OK not to be involved in everything helped to ease his worries.
“When I did the tour and they showed everything, it was kind of overwhelming and I felt like I have to do every single thing here,” Campbell said. “But now, I feel more like I can do something I feel passionate about and will excel at. It’s a bit more relieving.”
Randy Williams, vice president and associate provost for inclusive excellence, provided an overview of the Class of 2026 that includes 1,712 first-year students and Elon’s new 100 transfer students who as a group come from 45 states and 24 countries. Eighty-five students in the incoming cohort are siblings of current or former Elon students, 35 are the children or grandchildren of alumni, and 168 are first-generation college students.
Although impossible to fully depict the diverse and talented pool of students in the Class of 2026, Williams highlighted four students who are shining examples.
Nailah Ware ’26 from Crofton, Maryland, was the first applicant for the fall entering class. As a member of the Equity Leadership Council in high school and founding her own production company, Ware plans to combine those experiences as a music production and recording arts major to advance equity in the music industry.
Soriah Rodriguez Smith ’26 of Winter Park, Florida, balanced multiple commitments in high school. She was enrolled in an international baccalaureate program, was a member of a highly competitive varsity water polo team and worked two jobs. Smith is pursuing a psychology major with a future goal of becoming a physician assistant.
Erik Mason ’26 of Ardsley, New York, is pursuing an entrepreneurship degree at Elon and launched his own business as a high school freshman. Through his company, Divine Sneakers, Mason invests in and sells rare collectible sneakers, using the proceeds from his business to support youth organizations in White Plains. Mason was also a three-sport varsity athlete and coach for youth basketball.
Grace Muschett ’26 from Fairfield, Connecticut, played varsity lacrosse in high school and served as youth basketball coach. She developed a passion for teaching and learning as a volunteer with Scholars & Athletes Serving Others and will be turning that into a career, pursuing a degree in early childhood education as a Teaching Fellow at Elon.
“Nailah, Soriah, Erik and Grace, we welcome you to Elon and cannot wait to see what you and your classmates achieve,” Williams said. “While these four students are remarkable, each one of the new students seated before us today arrives with significant accomplishments and abilities.”
Drawing inspiration from the university’s Common Reading, “Factfulness,” President Book spoke about the hope she has in the Class of 2026. Conducting a human bar graph, Book illustrated the overwhelming positivity in the world that often goes overlooked. Hans Rosling, the author of “Factfulness,” argues that our instincts tend to be negative when things are more positive than they seem. Rosling doesn’t encourage us to ignore the negative things in life but to also focus on the improvements that have been made, Book said.
“This mindset … recognizes that each of us has the ability to advance and change the world. Things can get better, things are getting better,” Book said. “I like that view. It gives me hope and reminds me of our mission and the power of an Elon education.”
Faculty excellence was highlighted during the convocation ceremony noting Elon’s ranking as No. 1 in the country for undergraduate teaching. Faculty recognized this spring for excellence as mentors, scholars and community leaders were also highlighted, including Senior Lecturer in Religious Studies LD Russell, who was Associate Chair of the Department of Strategic Communications Vanessa Bravo, Associate Professor of Public Health Studies Stephanie Baker and Associate Professor of Anthropology Mussa Idris.
“I am proud of our Elon faculty and I know you will find them to be exceptional teachers and mentors during your time at Elon,” said Raghu Tadepalli, interim provost and dean of the Love School of Business.
Deandra Little, associate provost for faculty development, told students, “Over the next four years, you will get to know many of the faculty seated around you today. As you join the university, it’s important to get a sense of how you will work with them.”
This guidance stuck with Kate Becksvoort ’26, who was first uneasy about the relationship she would have with her professors. But after hearing about how helpful each faculty member is with all Elon students, regardless of discipline, Becksvoort was reassured that the faculty will be a huge resource for her to rely on during her four years at Elon.
“Everyone emphasized that there’s going to be a huge support system here and everyone cares about your well-being, where you go and how you’ll succeed,” she said.
The ceremony included a special musical performance from Travis Foust ’23 and Cecily Weinstein ’23 as well as the singing of Elon’s alma mater led by Polly Cornelius, senior lecturer in music. Students then processed through lines of faculty and staff members and received an acorn.
A treasured tradition of Elon for more than two decades, the receiving of an acorn at the New Student Convocation symbolizes intellectual and personal growth over the next four years. The next time the Class of 2026 will gather as a group Under the Oaks will be at their Senior Baccalaureate ceremony four years from now to receive an oak sapling – driving home the transformation they underwent at Elon.
“I feel so passionately that the world needs Elon graduates,” Book said. “We say that a lot around here, Class of 2026. Because the world and our country need the values an Elon education nurtures – respect for the richness our differences bring, the value of living a life of contribution and hard work, and making a difference in the lives of others – the values that will unfold for you during the next several years of engaged learning. You are our hope, our hope for a smarter, stronger and kinder world.”
Events Student Experience
New Student Orientation Parents
More than 1,700 members of the Class of 2026 and 100 transfer students gathered with their families and friends Under the Oaks on Saturday to participate in this university tradition.
Changes in the weather forecast will allow for the event for new Elon students to be held outdoors in this iconic university location.
The Elon University community welcomed the 1,700 students from the Class of 2026 and more than 100 transfer students with open arms as these newest Elon students moved into their new homes.
Coming from 23 countries, Elon welcomed the Class of 2026 international students with a breakfast in McKinnon Hall on Thursday.
Get more Elon news delivered to your inbox.
Loading
© 2022 Elon University | All Rights Reserved
A project uses sounds from the Western Ghats to understand the changes sweeping through the landscape – The Hindu
The year was 2020, and the pandemic was forcing musicians across the world to move away from live performances and record-production to projects centred on self-reflection and discovery. Musicians Krishna Jhaveri and Sanaya Ardeshir were no different.
Over a three-month-long journey, Jhaveri and Ardeshir travelled through the Western Ghats, armed with field recorders, and listened to the sounds of one of the world’s oldest mountain ranges. During their journey, they learned of the mountains’ unique ecology and through ‘deep listening’, were able to decipher the messages the inhabitants of the landscape were communicating to them, such as change in land use patterns, deforestation and habitat loss.
Recording in the wild. | Photo Credit: Special Arrangement
Through their project, ‘Ears to the Ground’ (www.ears2theground.org), Jhaveri and Ardeshir, who have been involved in the music industry for over a decade, recorded the natural sounds of the wild, and interviewed people living in the Western Ghats. They had no clear structure to the project except to find interesting ‘soundscapes’.
“All we thought was that we would record circadian rhythms. We riffed off the idea during the pandemic that if there was ever a time to travel through South India, just seeing what comes out of recording sounds, then this would be the time to do that,” says Ardeshir. The sounds of the forests, coupled with the context provided by the mountains’ human inhabitants, helped Jhaveri and Ardeshir discover the anthropogenic effects of human communities on the forests.
“Listening to people’s stories we could tell that the soundscape was changing, and climate change was something you could actually hear. We slowly realised that people were either speaking to us about sounds that were disappearing or new sounds emerging,” says Jhaveri.
The recordings were done through two different methods: when the duo was present, or when their field recorders were left in wild spaces.
“So you hear birds, and insects, maybe an animal occasionally and suddenly a chainsaw, or a plane flying overhead washing over the soundscape. Then you realise that the human footprint is ubiquitous, and it prompts very profound questions about what the word ‘wild’ actually means. Does it even exist?” Jhaveri ponders.
“I like the word ‘wild’ to distinguish between the natural and unnatural. I mean, is a city unnatural? It’s probably as natural as a biodiversity hotspot. The soundscape includes humans,” he says.
Visitors to the ‘Ears to the Ground’ website can listen to field recordings and interviews from five locations in the Western Ghats: Vagamon, Shoranur, Kerehaklu, Madikeri and Agumbe. But the project will continue to evolve. “The people we interviewed said they were becoming more attuned to listening. For instance, we have an audio recording of a person describing the sound of langurs suddenly appearing where they never had previously,” says Ardeshir.
The interviews also revealed to them that interventions could minimise human impact on these landscapes. “For instance, we met Ajoy and Pranoy Thipaiah in Kerehaklu; they were growing shade-grown coffee, and have designed pathways for large animals such as elephants and gaur to make their way through their land. While in Madikeri, Abhishek Jain, an organic farmer and naturalist, has invested in a reforestation programme, and has accepted a lower yield of coffee in exchange for a healthier ecosystem,” says Jhaveri.
They are also looking at ways to contribute to the conservation of the Western Ghats through their work. “We are not conservationists. We do not have a background in working on environmental issues, but we believe that sound and art have the ability to enrich messages with a lot more impact,” says Ardeshir.
The ethical questions about using sounds recorded from these locations have also been on their minds. “We are trying to involve avian ecologists and other people so that our recordings are not used in an extractive way, and the revenue we generate is directed back to the Western Ghats where they can be used, maybe in reforestation programmes,” says Jhaveri. They are now involving ecologists from Valparai, including those working with Project Dhwani, which records the sounds of biodiversity in natural, unrestored forests.
Jhaveri hopes that ultimately, Ears to the Ground would ‘switch on’ people’s listening, be it in an urban, rural or wild listening environment. “There is inherent value to listening. It has wider implications, so the main takeaway is to really listen.”
“In a very visually-dominating culture, we found that there are parallels between advertising, consumerism and a hyper fast-paced lifestyle. We hope that through the project sound connects people to themselves and to each other through compassionate listening; the idea is that sound can connect you to the environment,” says Ardeshir.
Our code of editorial values
Printable version | Aug 20, 2022 2:51:14 pm | https://www.thehindu.com/sci-tech/energy-and-environment/a-project-uses-sounds-from-the-western-ghats-to-understand-the-changes-sweeping-through-the-landscape/article65697306.ece
© THG PUBLISHING PVT LTD.
Judge orders Eastman to testify on Trump allegations tied to Georgia elections – Santa Fe New Mexican
Advertisement
Courts Reporter
A state district judge in Santa Fe has ordered John Eastman, an attorney for former President Donald Trump, to appear before a Georgia grand jury later this month to testify about allegations of coordinated attempts to unlawfully alter the outcome of Georgia’s 2020 elections.
Eastman, who has a home in Santa Fe, made headlines nationwide in June when federal agents confiscated his cellphone outside a restaurant in the city.
He was critical in the former president’s attempts to convince Republican Party members in at least a half-dozen states, including New Mexico, to replace Electoral College votes won by Biden with electors who would back Trump.
Eastman’s attorney, Joseph Gribble, argued in a court hearing Wednesday before Judge Mary Marlowe Sommer that Eastman — part of a legal team representing Trump in his bid to remain in office after the 2020 election — should not be compelled to appear before the grand jury.
His client does not fit the definition of a necessary and material witness, in part because he likely would exercise his Fifth Amendment right to remain silent on most questions due to the attorney-client privilege created by his position, Gribble argued.
The attorney also suggested the judge delay granting the motion to allow Eastman more time to “see what type of questions they would be asking” so he can talk to his clients or get an order of protection from the court, according to an audio recording of the hearing.
Gribble said the petition from Georgia’s Fulton County Superior Court to compel Eastman’s attendance at the grand jury didn’t protect Eastman from “any potential criminal recommendations” that could be generated from the grand jury investigations.
“All Georgia wants to do is fly him out there for him to assert these rights; that is just undue hardship,” Gribble argued. “Mr. Eastman can assert his Fifth Amendment right, and if we are proffered proper questions, assert his attorney-client privilege, and he doesn’t need to be forced, to be dragged, to Georgia to do that.”
He added, “This court has the power to protect a New Mexico resident from having to go and provide that information.”
First Judicial District Attorney Mary Carmack-Altwies told Sommer the Georgia court had sent plane tickets and a check for Eastman to reduce any burden he would face testifying before the special grand jury. Eastman would “not be having to pay a penny of his own money” to comply with the order, she added.
Sommer granted the petition, saying Eastman appeared to fit the definition of a material witness in the case and that any issue regarding attorney-client privilege or his right to remain silent should be addressed by the Georgia court.
As for “undue hardship,” the judge said, the phrase usually brings to mind a disability, financial issue or medical condition, none of which seemed to apply in Eastman’s case.
“What he’s essentially arguing is it’s not convenient, and I do not think that inconvenience rises to the level of undue hardship,” Sommer said.
Gribble did not respond to a call and email seeking comment Thursday.
Wednesday’s proceeding was — through a series of apparent court system mix-ups — held in relative secrecy.
The hearing opened with Carmack-Altwies trying to clear the courtroom, saying: “This was not filed under seal, but it is supposed to be a closed hearing, and I think there [are] some people in this room that would not qualify to be in here if this was a fully closed hearing.”
Deputy District Attorney Jennifer Padgett Macias said the case had been listed as “confidential” on the court’s docket.
But Sommer said that was incorrect. “They just didn’t know how to file it.”
First Judicial District Court Executive Officer Kathleen Vigil said Thursday the Clerk’s Office had coded the case in a way that automatically sealed it. She said she was unable to get more information on why it was coded for seal.
With the court file under seal, details on the hearing were hard to come by.
The New Mexican inquired about Wednesday’s hearing with the Administrative Office of the Courts a day earlier and was told the hearing had been vacated and reset for Aug. 26.
Administrative Office of the Courts spokesman Barry Massey on Thursday emailed an audio recording of the hearing, which he said “got reinstated” and had taken place Wednesday after all.
Sommer wrote in an email Thursday she had vacated and rescheduled the hearing because of a misunderstanding over whether Eastman had been properly notified about the case.
After receiving a response to the motion from Eastman’s attorney, she wrote, she set the hearing again for Wednesday.
Courts Reporter
{{description}}
Email notifications are only sent once a day, and only if there are new matching items.
Advertisement
Sorry, there are no recent results for popular videos.
Get up-to-the-minute news sent straight to your device.
Sorry, an error occurred.
Stay informed of the latest local news by receiving emails as soon as news is posted online.
Receive a list of headlines from the latest edition of The New Mexican in your inbox every morning.
Contests and special offers from The Santa Fe New Mexican and advertising partners.
Thank you .
Your account has been registered, and you are now logged in.
Check your email for details.
Invalid password or account does not exist
Submitting this form below will send a message to your email with a link to change your password.
An email message containing instructions on how to reset your password has been sent to the e-mail address listed on your account.
Secure & Encrypted
Secure transaction. Cancel anytime.
Thank you.
Your purchase was successful, and you are now logged in.
A receipt was sent to your email.
Generation of realistic synthetic data using Multimodal Neural Ordinary Differential Equations | npj Digital Medicine – Nature.com
Thank you for visiting nature.com. You are using a browser version with limited support for CSS. To obtain the best experience, we recommend you use a more up to date browser (or turn off compatibility mode in Internet Explorer). In the meantime, to ensure continued support, we are displaying the site without styles and JavaScript.
Advertisement
npj Digital Medicine volume 5, Article number: 122 (2022)
2
Metrics details
Individual organizations, such as hospitals, pharmaceutical companies, and health insurance providers, are currently limited in their ability to collect data that are fully representative of a disease population. This can, in turn, negatively impact the generalization ability of statistical models and scientific insights. However, sharing data across different organizations is highly restricted by legal regulations. While federated data access concepts exist, they are technically and organizationally difficult to realize. An alternative approach would be to exchange synthetic patient data instead. In this work, we introduce the Multimodal Neural Ordinary Differential Equations (MultiNODEs), a hybrid, multimodal AI approach, which allows for generating highly realistic synthetic patient trajectories on a continuous time scale, hence enabling smooth interpolation and extrapolation of clinical studies. Our proposed method can integrate both static and longitudinal data, and implicitly handles missing values. We demonstrate the capabilities of MultiNODEs by applying them to real patient-level data from two independent clinical studies and simulated epidemiological data of an infectious disease.
Patient-level data build the foundation for a plethora of healthcare research endeavors such as drug discovery, clinical trials, biomarker discovery, and precision medicine1. Collecting such data is extremely time-consuming and cost-intensive, and additionally access-restricted by ethical and legal regulations in most countries. Individual organizations, such as hospitals, pharmaceutical companies, and health insurance providers are currently limited in their ability to collect data that are fully representative of a disease population. This issue is especially pronounced in clinical studies, where patients are usually recruited based on predefined inclusion and exclusion criteria that introduce cohort-specific statistical biases2. These biases, in turn, can negatively impact the generalization ability of machine learning models, since the usual i.i.d. assumption is violated3. A naive idea to counteract this issue might be to build up large data repositories pooling diverse clinical studies from several organizations. However, here, a major obstacle is that sharing patient-level data across different organizations is exceedingly difficult due to legal restrictions, as formulated, for example, in the General Data Protection Rule of the European Union.
The idea we propagate in this paper is to learn a continuous-time generative machine learning model from clinical study data. Given the distribution of the real training data was appropriately learned by such a model, the generated synthetic datasets maintain the real data signals, such as variable interdependencies and time-dependent trajectories. Furthermore, these synthetic datasets can overcome crucial limitations of their real counterparts like missing values or irregular assessment intervals, hence opening the opportunity to make at least subsets of variables from different studies statistically comparable. A further strong motivation for generating synthetic datasets is the aim to use the generated data as an anonymized version of its real-world counterpart and thereby mitigate the increased restrictions for sharing human data4,5,6. However, synthetic patient-level datasets open opportunities that reach far beyond data sharing. For example, trained generative models could be used for synthesizing control arms for clinical trials based on data from previously conducted trials, or from real-world clinical routine data7. This helps addressing major ethical concerns in disease areas, such as cancer, where it is impossible to leave patients untreated. Both, the American Food and Drug Administration and the European Medicines Agency have recognized this issue and taken initiatives to allow for synthetic control arms7.
Over the last years, generative models (mostly generative adversarial networks [GANs]) have found notable success, mostly in the medical imaging domain8,9,10,11,12,13. However, GANs are often found to show a collapse in the statistical mode of a distribution, which raises concerns regarding coverage of the real patient distribution by synthetic data. Moreover, these methods are not necessarily suited to cope with the complex nature of clinical data collected in observational, longitudinal cohort studies, which is the main focus of our work: In addition to the previously mentioned issue of irregular measurement frequencies and missing values not at random (e.g., due to participant drop-out), clinical studies often comprise several modalities combining time-dependent variables (e.g., measures of disease severity) and static information (e.g., biological sex). One approach specifically designed for the joint modeling and generation of multimodal, time-dependent, and static patient-level data containing missing values is the recently introduced Variational Autoencoder Modular Bayesian Networks (VAMBN)4. However, VAMBN only operates on a discrete time scale while relevant clinical indicators such as, for example, disease progression expressed through a cognitive decline or rising inflammatory markers, are intrinsically time continuous. Recently, Neural Ordinary Differential Equations (NODEs) have been introduced as a hybrid approach fusing neural networks and Ordinary Differential Equations (ODEs)14. While NODEs are time continuous and thus enable smooth interpolation between observed data points and extrapolation beyond the observations in the data, they are not able to integrate static variables.
In this work, we present the Multimodal Neural Ordinary Differential Equations (MultiNODEs) as an extension of the NODEs. MultiNODEs allow learning a generative model from multimodal longitudinal and static data that may contain missing values not at random. To demonstrate MultiNODEs’ generative capabilities, we applied the model to clinical, patient-level data from an observational Parkinson’s disease (PD) cohort study (the Parkinson’s Progression Markers Initiative [PPMI]15) and, additionally, a longitudinal Alzheimer’s disease (AD) data collection (National Alzheimer’s Coordination Center [NACC]16). We compared the generated trajectories and correlation structure with the real counterpart. In this context, we additionally evaluated MultiNODEs’ performance against the previously published VAMBN approach. Furthermore, we assessed MultiNODEs’ interpolation and extrapolation performance. Finally, we investigated the influence of sample size, noisiness of the data, and longitudinal assessment density on the training of MultiNODEs in a systematic benchmark on data simulated from a mathematical model well-known in the epidemiology field.
MultiNODEs represent an extension of the original NODEs framework14 that overcomes the limitations of its predecessor such that an application to incomplete datasets consisting of both static and time-dependent variables becomes feasible. Conceptually, MultiNODEs build on three key components (Fig. 1): (1) latent NODEs, (2) a variational autoencoder (more specifically a Heterogenous Incomplete Variational Autoencoder [HI-VAE], designed to handle multimodal data with missing values17), and (3) an implicit imputation layer18. The latent NODEs enable the learning and subsequent generation of continuous longitudinal variable trajectories. The longitudinal properties of the initial condition (i.e., the starting point for the ODE system solver of the latent NODEs) are defined by the output of a recurrent variational encoder that embeds the longitudinal input data into a latent space (Fig. 1, orange box). To allow for an additional influence of static variables on the estimation of the longitudinal variable trajectories, the second component, a HI-VAE, is introduced (Fig. 1, blue box). This component transforms the static information into a distinct latent space and the resulting embedding is used to augment the latent starting condition of the NODEs by concatenating the static variable embedding and the latent representation of the longitudinal variables (Fig. 1, “augmentation”). The HI-VAE component itself holds generative properties and conducts the synthesis of the static variables when MultiNODEs are applied in a generative setting. Conclusively, MultiNODEs integrate static variables (e.g., biological sex or genotype information) both to inform the learning of longitudinal trajectories, and in the generative process. Finally, to mitigate the original NODEs’ incapability of dealing with missing values, we introduced the imputation layer which implicitly replaces missing values during model training with learned estimates (Fig. 1, green box). For further details on the model architecture, training, and hyperparameter optimization, we refer to the Method section and Supplementary material, respectively.
Blue box: HI-VAE for the encoding and generation of static variables. Orange box: NODEs that learn and generate longitudinal trajectories. Green box: the imputation layer that can handle missing data implicitly during model training.
Generating synthetic data using MultiNODEs starts by randomly sampling a latent representation for both the static and longitudinal variables, respectively. The longitudinal variables in data space are then generated by first constructing the initial conditions of the latent ODE system (i.e., concatenating the static latent representation to the longitudinal one), followed by solving the ODE system given these initial conditions, and finally by decoding the result into data space. The static variables are generated by directly transforming their sampled latent representation into data space using the HI-VAE decoder.
MultiNODEs support two different approaches for the initial sampling of the latent representations, namely sampling from the prior distribution employed during model training and sampling from the learned posterior distribution of the input data.
During the posterior sampling procedure, the reparameterization trick19 is applied to draw a latent representation from the posterior distribution learned from the training data. The amount of noise added in this process can be tuned, whereas greater noise will lead to a wider spread of the generated marginal distributions of the synthetic data. Alternatively, the latent representations can be sampled from the prior distributions imposed on the latent space during variational model training. We ensure statistical dependence between static and longitudinal variables by drawing their values from a Bayesian network that connects both latent representations such that the longitudinal variables are conditionally dependent on the static variables. More detailed descriptions of both generation procedures are provided in the Method section.
We applied MultiNODEs to longitudinal, multimodal data from two independent clinical datasets with the goal of generating realistic synthetic datasets that maintain the real data properties. Details about the data preprocessing steps are described in the Supplementary material.
The first dataset was the PPMI, an observational clinical study containing 354 de-novo PD patients who participated in a range of clinical, neurological, and demographic assessments which form the variables of the dataset. In total, a set of 25 longitudinal and 43 static variables was investigated.
Furthermore, as a second example, we applied MultiNODEs to longitudinal, multimodal data from the NACC. NACC is a database storing patient-level AD data collected across multiple memory clinics. After preprocessing, the dataset used in this study contained 2284 patients, and a set of three longitudinal and four static variables was investigated.
In the following sections, we will focus on the results achieved on the PPMI data and refer to the equivalent experiments based on the NACC data that are presented in the Supplementary material.
We applied prior as well as posterior sampling for comparison purposes. With each method, we generated the same number of synthetic patients as encountered in the real dataset to allow for a fair comparison. To assess whether the generated data followed the real data characteristics, we conducted thorough comparisons of the marginal distributions using qualitative and visual assessments and further, quantitatively compared the Jensen–Shannon divergence (JS-divergence) between the generated data and real distributions. The JS-divergence is bound between 0 and 1 with 0 indicating equal distributions. In addition, we investigated the underlying correlation structure of the measured variables. Finally, we trained a machine learning classifier (Random Forest) that evaluated whether real and synthetic patients showed similar clinical characteristics when compared to real healthy control individuals from their respective studies. Across all these aspects, we evaluated MultiNODEs’ performance in comparison to the previously published VAMBN approach4.
The synthetic data generated using MultiNODE generally exhibited marginal distributions that bore high similarity to their corresponding real counterparts (Fig. 2, Supplementary Table 1, and Supplementary Fig. 1; equivalent figures for the NACC data are presented in Supplementary Fig. 4). The average JS-divergences between the real and synthetic distributions calculated across all variables and timepoints amounted to 0.018 ± 0.015 and 0.011 ± 0.009 for the PPMI data generated from the prior and posterior, respectively. For NACC the average JS-divergence was 0.071 ± 0.055 and 0.029 ± 0.031 for prior and posterior sampling, respectively. With respect to PPMI, data generation from the posterior distribution resulted in synthetic data that resembled the real data significantly closer than those generated from the prior distribution (Mann–Whitney U test, p < 0.02).
Mean, standard deviation, and KL-divergence for the displayed variables can be found in Supplementary Table 1. Equivalent results for the NACC data are presented in Supplementary Fig. 5. a Time-dependent variable “SCOPA” at month 12. b Time-dependent variable “UPDRS2” at month 24. c Static variable “Aβ.42”. d Categorical static variable “Handedness”.
Compared to VAMBN, the prior sampling method seemed to be inferior with respect to the average JS-divergence when using NACC (U test, p = 0.038). However, no statistically significant difference in the performance of VAMBN compared to MultiNODE’s posterior sampling could be observed (U test, p = 0.80). For PPMI, no significant differences were found between VAMBN and any of MultiNODEs’ generation approaches (U test, p = 0.31 for the prior approach; U test, p = 0.24 for the posterior).
In order to evaluate whether MultiNODEs learned not only to reproduce univariate distributions but actually captured their interdependencies accurately, we compared the correlation structure of the generated data to that of the real variables. Visualizations of the Spearman rank correlation coefficients showed that both the prior and posterior sampling generated synthetic data which successfully reproduced the real variables’ interdependencies (Fig. 3). Comparing the results against VAMBN-generated data revealed that both generation procedures of MultiNODEs were significantly better at reproducing the real data characteristics: the Frobenius norm of real data correlation matrix resulted in 45.3, and with a Frobenius norm of 25.66 the VAMBN-generated data placed substantially further from the real data than the MultiNODEs approaches with 62.63 and 56.47 for the prior and posterior sampling, respectively. This shows that MultiNODEs slightly overestimated the present correlations, while VAMBN underestimated them. Concordantly, the relative error (i.e., the deviation of the respective synthetic dataset’s correlation matrix from the real one normalized by the norm of the real correlation matrix), was 0.81, 0.62, and 0.46, respectively, for VAMBN and MultiNODEs’ prior and posterior sampling, leaving MultiNODEs with a substantially lower error than the VAMBN approach.
Equivalent results for the NACC data are shown in Supplementary Fig. 6. a Real data. b Posterior sampling from MultiNODEs. c Prior sampling from MultiNODEs. d VAMBN-generated data.
To evaluate whether the generated synthetic patients could be reliably used in a machine learning context, we built a Random Forest classifier that aimed to distinguish between healthy individuals and diseased patients. The classifier was trained within a five-fold cross-validation scheme once using real and once using synthetic diseased patients. In addition, we trained a classifier on each respective synthetic dataset (comprising synthetically generated diseased and healthy subjects) and evaluated their performance on the real data (Table 1). As predictors, we used clinical symptoms and genetic markers that are characteristic of the disease in question. For PD (PPMI), these were the UPDRS scores that describe a series of motor and non-motor symptoms commonly encountered in PD patients, for AD (NACC), we predominantly used cognitive assessments and a genetic risk factor. Technical details about the classifiers can be found in the Supplementary material. Distinguishing real patients from healthy control subjects was possible with a 10 times repeated five-fold cross-validated performance of 0.97 ± 0.02 area under the receiver operator curve (AUC) and 0.90 ± 0.01 AUC for PPMI and NACC, respectively. On PPMI, all evaluated generative methods achieved almost equal performance, indicating that clinical characteristics of synthetic patients followed the same patterns as in real patients. In addition, the most relevant features were the same across the real and all synthetic data-trained classifiers (Supplementary Fig. 13).
For NACC, some deviations were found between a classifier’s cross-validated performance on real data and the synthetic-data-based performances. Here, MultiNODEs’ posterior and VAMBN showed similar deviations in opposite directions, with the posterior slightly overperforming and VAMBN slightly underperforming. The performance on the data generated via MultiNODE’s prior sampling method deviated the most (Table 1). When trained on synthetic data and evaluated on real data, all trained classifiers underperformed compared to classifiers trained on real data. The feature importances of predictors were highly similar between the real data-trained and the respective synthetic data-trained classifiers.
One particular strength of MultiNODEs, that sets it apart from alternative approaches such as VAMBN, is its ability to model variable trajectories in continuous time. The latent ODE system allows for the estimation of variable trajectories at any arbitrary timepoint and thereby opens possibilities for (1) the generation of smooth trajectories, (2) overcoming panel-data limitations through interpolation, and finally, (3) extrapolation beyond the time span covered in training data themselves. Again, we evaluated these capabilities based on the PPMI and NACC datasets (for brevity, NACC results are presented in the Supplementary material). For the following, we only focused on the MultiNODE posterior sampling approach to generate synthetic subjects.
Comparing the median trajectories of variables from the real data to those generated using MultiNODEs revealed that MultiNODEs accurately learned and reproduced the longitudinal dynamics exhibited in the real data (Fig. 4). Generation from both the prior and posterior distribution led to synthesized median trajectories that closely resembled the real median trajectories. Equivalently, also the 97.5% and 2.5% quantiles of the synthetic data approximated the corresponding real quantiles closely, indicating a realistic distribution of the synthetic data across the observed timepoints. This observation held true for most of the time-dependent variables (plots for all variables are linked in the Supplementary material).
Additional examples are provided in Supplementary Fig. 2. A corresponding example for the NACC dataset is shown in Supplementary Fig. 7. a–d depict different longitudinal variables from the PPMI dataset.
We further assessed the interpolation and extrapolation capabilities of MultiNODEs. For interpolation, one timepoint was excluded from model training and subsequently, data were generated for all timepoints including the one left-out. Contrasting the interpolated/imputed values against the corresponding real values showed that MultiNODEs accurately reproduced the longitudinal dynamics of a variable, even for unobserved timepoints (Fig. 5a, c). In this context, we further compared the interpolated values against synthetic data that was generated based on the complete real data trajectory. We observed that the mean JS-divergence calculated across all variables between the interpolated data and the real data was slightly higher (0.025 ± 0.011) than that of the real data and the synthetic data generated after training MultiNODEs on the complete trajectory (0.016 ± 0.011). Similarly, the relative error between the interpolated correlation matrix and the real data was again only marginally higher than between the complete data and the real data (0.48 and 0.46, respectively; Supplementary Fig. 4).
The black box indicates the interpolated and extrapolated sections. Plots for additional variables are presented in Supplementary Fig. 3. A corresponding example for the NACC dataset is shown in Supplementary Fig. 8. a Interpolation of the UPDRS1 variable at month 24. b Extrapolation of the last five assessments of the UPDRS1 variable. c Distribution of the interpolated values for UPDRS1 at visit 24. d Distribution of the extrapolated values for UPDRS1 at month 42.
In order to test MultiNODEs’ extrapolation capabilities, only the first 24 months of assessment follow-up and the static variables were used during model training. The trained model was then applied to generate data for the remaining, left-out timepoints of the longitudinal variables. In this course, 77 values were extrapolated while not every variable had the same number of follow-up assessments after month 24. Comparing the extrapolated synthetic data to the left-out real data demonstrated reliable extrapolation beyond the training data (Fig. 5b, d). As in the interpolation setting, we also compared the average JS-divergence between the extrapolated data and the real data with that between the real data and synthetic data that were generated after training MultiNODEs on the complete trajectory. As expected, we could see a larger difference between the JS-divergences compared to the interpolation setting with 0.037 ± 0.024 for the extrapolated data and 0.016 ± 0.009 for the synthetic data based on the complete trajectory. The correlation structure in the extrapolation culminated in a relative error of 0.64 compared to 0.46 when using the complete trajectory for training MultiNODEs (Supplementary Fig. 4).
In addition, the marginal distributions at both the interpolated and extrapolated timepoints also followed those of the real data (Fig. 5c, d).
To explore the learning properties of MultiNODEs more systematically, we investigated how alternating training conditions with respect to measurement frequency, sample size, and noisiness of the data influence MultiNODEs’ generative performance.
The benchmarking data was simulated via the well-established Susceptible-Infected-Removed (SIR) model that is often used to describe the spread of infectious diseases and follows a highly nonlinear structure: Let S(t) be the number of susceptible individuals at a timepoint t, I(t) be the number of infectious individuals at a timepoint t and R(t) be the number of removed or recovered individuals at a timepoint t. With β as transmission rate, γ as mean recovery/death rate, and N = S(t) + I(t) + R(t) as fixed population size the SIR model can be defined by the ODE system presented in Eq. (1):
Details about the SIR parameter settings are described in the Supplementary material.
As baseline settings for each investigation, we simulated 1000 data points with 10 equidistant assessment timepoints each, distributed over a span of 40 time intervals, and added 5% Gaussian noise to each measurement. That means we added a normally distributed variable with the standard deviation set to 5% of the theoretical range of each of the variables S(t), I(t), and R(t). During the benchmarking, we individually alternated the sample size, timepoints, and noise level. For the timepoint investigation, we compared MultiNODEs trained on 5, 10, and 100 equidistant assessments; for the sample size we considered 100, 1000, and 5000 samples; and for the noise level, we tested 50%, 75%, and 100% of the maximum encountered value added as noise.
Alternating the amount of equidistant, longitudinal timepoints exposed a strong dependency of MultiNODEs on the longitudinal coverage of the time-dependent process (Fig. 6a). While the general trends in the data were appropriately learned for all explored assessment frequencies, the position of the observations in time influenced how close the learned function approximated the true data-underlying process. Especially the peak of the “Infected”-function represented a challenge for MultiNODEs if no data point was located close to it (Fig. 6a, “Infected”). Similarly, the start of the decline in the “Susceptible”-function and the incline in the “Removed”-function were shifted, depending on the positioning of measurements. In conclusion, and as expected, a higher observation frequency of the data-underlying the time-dependent process significantly increased the fit of MultiNODEs to the process, although, general trends could already be approximated for lower assessment frequencies.
Each panel (a–c) represents the evaluation of another parameter (assessment frequency, sample size, and noise level, respectively).
Investigating the effect of the sample size on training MultiNODEs, we observed that an increase of the sample size led to an expected improvement of the model fit to the SIR dynamics (Fig. 6b). While the general trends could again be learned from limited data (n = 100), sample sizes of 1000 or 5000 substantially reduced the model’s deviation from the true SIR model. With 1000 samples, the learned dynamic is less stable than when trained on 5000 samples, where a smooth dynamic was learned that closely resembled the true underlying process. In conclusion, MultiNODEs can already learn longitudinal dynamics based on only a few data points, however, they tend to underfit under these circumstances and benefit from larger sample sizes.
Adding an increasing noise level to the SIR training data revealed that MultiNODEs remain very robust (Fig. 6c). Only when introducing 100% of the maximal encountered value as additional noise, a clear deviation from the underlying true model could be observed.
In this work, we presented MultiNODEs, a hybrid AI approach to generate synthetic patient-level datasets. MultiNODEs are specifically designed to consider the characteristics of clinical studies, extend its predecessor, the Neural ODEs, and enable the application of the latent ODE system to multimodal datasets comprising both time-dependent and static variables with values missing not at random. MultiNODEs learn a latent, continuous time trajectory from observed data. This concept fits well with processes like disease progression, where relevant observations (e.g., biomarkers and disease symptoms) only indirectly mimic the true, underlying disease mechanism. Consequently, MultiNODEs are well suited for an application to heterogeneous datasets holding complex signals as encountered, for example, in biomedical research.
Our evaluations showed that MultiNODEs successfully generated complex, synthetic medical datasets that accurately reproduced the characteristics of their real-world counterparts. In a direct comparison MultiNODEs’ outperformed the state-of-the-art VAMBN approach, most notably with respect to the integrity of the correlation structure. This finding implies that the single data instances generated using MultiNODEs exhibit more realistic properties and that the real data characteristics are not only reproduced at the population level. Out of MultiNODEs two generative methods, the posterior sampling expectedly led to more realistic synthetic patients; however, generating from the prior distribution comes with the benefit that the model itself can be shared and used for data generation without needing any real data points in the process.
Machine learning classifiers that discriminated between real healthy controls and diseased subjects showed almost equal performance when trained on data from synthetic and real diseased subjects, respectively. Here, we only observed small deviations from the performance on real data for the NACC dataset, where classifiers trained and tested on synthetic patients and real healthy controls within a cross-validation setting showed a slightly increased performance to those trained on real data. Interestingly, at the same time, we found a lower prediction performance compared to real data when we trained on synthetic subjects and evaluated on the real data. A possible explanation is that synthetic data can contain noise that is introduced during the generation of synthetic data points (e.g., through overestimated correlations between variables). Therefore, synthetically generated diseased patients are better discriminated against real healthy controls than real diseased patients. At the same time, this situation leads to the fact that a classifier trained on synthetic data (synthetic patients as well as healthy controls) shows a slightly lower prediction performance on real data compared to a classifier trained entirely on real data. Altogether our results demonstrate that synthetically generated subjects share patterns of real patients, but they are not completely identical.
Besides the reproduction of marginal distributions and synthesis of realistic data instances, MultiNODEs most prominent strength lies in the generation of smooth longitudinal data. The latent ODE system allows MultiNODEs to learn dynamics that are continuous in time and cover the unobserved time intervals of real-world data. Here, both the prior and posterior sampling approach resulted in synthetic trajectories that obey real variables’ dynamics.
Furthermore, the time-continuous generative capabilities of MultiNODEs create opportunities to fill gaps in the real data through interpolation and go beyond the observation time by extrapolating the longitudinal dynamics. Hence, MultiNODEs could be used to support the design of longitudinal clinical studies, in which the maximum observation period, as well as visit frequency, is always a crucial decision to make. Here, the question of how patients might develop between two visits or after the last one determines the optimal follow-up time, to demonstrate, for example, the most significant treatment effect. Furthermore, synthetic disease trajectories generated based on data from one clinical study can be compared to those generated based on other studies, even if the visit intervals employed in the real studies were not identical.
Our benchmark experiments on the simulated SIR model data demonstrated that MultiNODEs are applicable under a variety of different data settings. While the general trends of a data-underlying process could already be learned from a relatively limited dataset, similar to any machine learning task, the accuracy and trustworthiness of the model critically depends on the available data. Especially for complex, nonlinear processes, a sufficiently high observation frequency should be considered. Here, the position of the observation timepoints relative to the true underlying process is crucial for MultiNODEs to accurately learn nonlinear dynamics. The sample size of the training data mainly impacts how well MultiNODEs fitted the data dynamics and we observed that lower sample sizes can lead to underfitting and rather rigid ODE systems. On the other hand, only severe noise levels led to a model deviation from the true data-underlying process, and thus, with respect to noise, MultiNODEs proved to be highly robust. In conclusion, MultiNODEs’ requirements toward the training data ultimately depend on the complexity of the data-underlying process, whereas the learning of more complex processes requires more frequent observations and larger sample size, while more linear systems can already be learned from rather limited datasets.
One limitation of MultiNODEs in their current form only allows static categorical variables. This is because the variational encoder for longitudinal data maps trajectories to a latent Gaussian distribution. Sampling from this distribution (even, if conditioned on the distribution of the static data) and decoding will result in real valued features rather than categorical ones. In future work, we will thus explore whether a recurrent version of the HI-VAE encoder can be used instead of a recurrent variational long-short term memory (LSTM) encoder.
In addition, MultiNODEs are sensitive to several hyperparameters that should be optimized for optimal performance. The training process and all relevant hyperparameters are explained in the Method section.
Synthetic data generated using models trained on sensitive personal information can bear a risk of information disclosure (e.g., attribute disclosure or dataset membership disclosure), if an attacker has information about properties of real patients that are similar to a synthetic subject. Therefore, before synthetic data are distributed, it must be assured that the probability of private information disclosure remains within task-appropriate boundaries20. Disclosure risk often stands in a direct trade-off with data utility and a sensible compromise should be taken balancing the two according to the application in question. Several approaches are described in the literature that can reduce the risk of information disclosure21, one of which is based on the concept of differential privacy4. MultiNODEs themselves provide a way to tune the deviation from the real data when sampling from the posterior distribution by changing the amount of noise injected in the latent space. We would like to mention that a rigorous quantification of the re-identification risk is a non-trivial and challenging task for its own requiring several assumptions and is thus beyond the scope of this paper.
Both datasets, namely PPMI and NACC, are well-known staples in their respective fields and can be accessed after successful data access applications. For PPMI see https://www.ppmi-info.org/. For NACC we refer to https://naccdata.org/. More details on the investigated variables are presented in the Supplementary material.
Both studies retrieved informed consent from their participants for data collection and sharing and followed the declaration of Helsinki to ensure ethical data collection. Both studies got ethical approval from their respective review boards. We followed their employed regulations and thus did not seek further ethical approval, as we did not work with human participants ourselves.
NODEs are a hybrid of neural networks and ODEs14. They can be seen as an extension of a ResNet22, which does not rely on a discrete sequence of hidden layers, but on a continuous hidden dynamical system defined by an ODE.
For 0 < t < M and (z_0 in R^D) the dynamics of the hidden layer of a NODE are given as Eq. (2).
where z(0) may be interpreted as the first hidden layer and z(T) as the solution to the initial value problem at timepoint T. Importantly, f is a feed-forward neural network parameterized by θ.
As demonstrated by the authors in their publication, NODEs can be trained as a continuous time Variational Autoencoder. The basic idea is to learn the initial conditions z0 of the dynamical system in Eq. (2) from observed time series data using a variational LSTM recurrent encoder23. Hence, Eq. (2) now describes the dynamics of a latent system, resulting in a classical state-observation model. Accordingly, a feed-forward neural network decoder is required to project the solution of Eq. (2) back to observed data at defined timepoints (Supplementary Fig. 10).
Overall NODEs are trained at once by maximizing the evidence lower variational bound (ELBO): let the training data be (D = { ( {x_{t_i}^n,t_i}) | n = 1, ldots ,N,i = 1, ldots ,M}), where N is the number of patients and (t_{i_1}, ldots ,t_{i_M}) the observed timepoints / patient visits. That means (x_{t_i}^n in R^p) is the p-dimensional vector of measurements taken for the nth patient at visit ti. The ELBO for NODEs is then given as Eq. (3).
where (pleft( {z_{t_0}^n} right) = Nleft( {0,I} right)), as usual. For details, we refer to Chen et al.14.
To handle missing values (potentially not at random) in longitudinal clinical data we build on our previously published work, in which we introduced an imputation layer to implicitly estimate missing values during neural network training18: let (A: = left{ {x_{t_i,j}^n{{{mathrm{|}}}}x_{t_i,j}^n {rm{is not missing}}} right}), 1A be the indicator function on set A with cardinality (left| A right|). The imputation layer can be defined as a data transformation (tilde x_{t_i,j}^n = x_{t_i,j}^n times 1_Aleft( {x_{t_i,j}^n} right) + b_{t_i,j} times left( {1 – 1_Aleft( {x_{t_i,j}^n} right)} right)), where parameters (b_{t_i,j}) are trainable weights. That means missing values in a patient’s data vector (x_{t_i,j}^n) are replaced by (b_{t_i,j}). The accordingly completed data is subsequently mapped through a recurrent neural network encoder to a static, lower dimensional vector, which is interpreted as the initial condition of the latent ODE system (Supplementary Fig. 11).
To learn parameters (b_{t_i,j}) the NODEs’ loss function needs to be adapted. More specifically, we use the modified ELBO criterion presented in Eq. (4).
where (hat x_{t_i,j}^n) denotes the reconstructed data. Note that we only aim for reconstructing the observed data, but not the imputed one. Due to the layer-wise architecture of a neural network (hat x_{t_i,j}^n) implicitly depends on (b_{t_i,j}).
In practice, we initialize (b_{t_i,j}) for neural network training as (frac{1}{N}mathop {sum }nolimits_{n = 1}^N x_{t_i,j}^n).
In addition to implicit missing value imputation, the second main idea of MultiNODEs is to complement NODEs with a HI-VAE encoder17 for static variables (Supplementary Fig. 11). A HI-VAE is an extension of a Variational Autoencoder that can implicitly impute missing values via an input drop-out model and handle heterogeneous multimodal data, including categorical data and count data, via an accordingly factorized generative model. In addition, a HI-VAE uses a Gaussian Mixture Model (GMM) as a prior distribution rather than a single Gaussian. We refer to Nazabal et al.17 for details.
The HI-VAE results in a lower dimensional latent representation zstat of static variables, which can be used to augment the initial conditions zt0 learned from time series data. Consequently, we arrive at the following formulation of the latent ODE system given in Eq. (5).
This approach resembles the Augmented Neural ODEs by Dupont et al.24. In contrast to our work, in their work no additional features were added during the augmentation step, i.e., zstat = 0. According to Dupont et al. the purpose of Augmented Neural ODEs is to smoothen f, whereas we focus here on multimodal data integration.
For training MultiNODEs, we have to jointly consider (ELBO_{IMP}^{NODE}) as well as (ELBO^{HI – VAE}). After bringing both quantities on a comparable numerical scale, we use a weighted sum as our final training objective (see Eqs. (6) and (7)):
Where β is a tunable hyperparameter. Details about hyperparameter optimization are described in the Supplementary material.
We tested two methods to generate synthetic subjects with MultiNODEs:
The first option is drawing a sample of latent static and longitudinal representations from the respective prior distributions (z_{t_0} sim Nleft( {0,I} right))and (z_{stat} sim GMMleft( pi right)). To assure that interdependencies between static and longitudinal variables are conserved, we model their joint distribution (Pleft( {z_{t0},z_{stat}} right)) using a Bayesian network. This network contains three nodes (random variables) representing (1) the GMM mixture coefficients π for the static data used by the HI-VAE, (2) the latent static representations (Z_{stat} = GMMleft( pi right)), and (3) the latent longitudinal representations (Z_{t0} = Nleft( {0,I} right)), respectively. The network is constrained such that directed edges can only go from si to Zstat and from there to Zt0. After randomly sampling a mixture component si from a multinomial distribution multinom(π), we can conditionally sample (z_{stat} sim Z_{stat} | s_i) and finally (z_{t0} sim Z_{t0} | Z_{stat}). Subsequently, we concatenate (z_0 = left[ {z_{t_0},z_{stat}} right]) into a vector forming the initial conditions for the latent ODE system, solve the ODE system, and decode the solution. We call this approach “prior sampling”.
A second option is to draw (z_{t_0}q( {z_{t_0}^n | {{x_{t_i}^n,t_i} }_i} ) = N( {lambda ( {x_{t_i}^n,t_i}),sigma ( {x_{t_i}^n,t_i})})) for the longitudinal data and (z_{stat}qleft( {z_{stat}^n{{{mathrm{|}}}}x_{stat}^n,pi } right) = Nleft( {lambda left( {x_{stat}^n,s^n} right),sigma left( {x_{stat}^n,s^n} right)} right),s^nCategoricalleft( {pi left( {x_{stat}^n} right)} right)) for the static data. That means we generate a blurred / noisy version of the original nth patient. We call this approach “posterior sampling” and recommend this sampling procedure for data generation. In our experiments, we doubled the posterior variance during sampling because we found the synthetic data otherwise to lie too close to the real data. Tuning the added noise can provide one option to balance identification risk versus data utility.
Synthetic data can not only be generated for observed visits, but also for definable timepoints in between (interpolation) and after the end of the study (extrapolation). This is possible because the latent ODE system is continuous in time.
Few steps are required to preprocess the clinical data before MultiNODEs can be applied. First, the data must be organized into a three-dimensional tensor of the shape samples × timepoints × variables for the longitudinal variables, and samples × variables for the static ones. Furthermore, the longitudinal variables are then transformed into a progression score by subtracting the baseline value and normalizing them by the standard deviation of this variable at baseline.
The relative error between correlation matrices is calculated as the norm of the matrix describing the difference between the real correlation matrix and synthetic data correlation matrix divided by the norm of the real correlation matrix.
Further information on research design is available in the Nature Research Reporting Summary linked to this article.
The PPMI dataset is available under: https://www.ppmi-info.org/. The NACC data are available under: https://naccdata.org/. The data are shared by the data owners after successful application. The data generated for this study cannot be shared by the authors due to the signed data usage agreements with the data owners of the corresponding real data (i.e., PPMI and NACC).
The code for MultiNODEs is available at https://github.com/philippwendland/MultiNODEs.
Fröhlich, H. et al. From hype to reality: data science enabling personalized medicine. BMC Med. 16, 150 (2018).
Article Google Scholar
Birkenbihl, C., Salimi, Y. & Fröhlich, H. Japanese Alzheimer’s Disease Neuroimaging Initiative; Alzheimer’s Disease Neuroimaging Initiative Unraveling the heterogeneity in Alzheimer’s disease progression across multiple cohorts and the implications for data‐driven disease modeling. Alzheimers Dement. 18, 251–261 (2022).
Article Google Scholar
Birkenbihl, C. et al. Differences in cohort study data affect external validation of artificial intelligence models for predictive diagnostics of dementia – lessons for translation into clinical practice. EPMA J. 11, 367–376 (2020).
Article Google Scholar
Gootjes-Dreesbach, L., Sood, M., Sahay, A., Hofmann-Apitius, M. & Fröhlich, H. Variational Autoencoder Modular Bayesian Networks for simulation of heterogeneous clinical study data. Front. Big Data 3, 16 (2020).
Article Google Scholar
Sood, M. et al. Realistic simulation of virtual multi-scale, multi-modal patient trajectories using Bayesian networks and sparse auto-encoders. Sci. Rep. 10, 10971 (2020).
CAS Article Google Scholar
Chen, R. J., Lu, M. Y., Chen, T. Y., Williamson, D. F. K. & Mahmood, F. Synthetic data in machine learning for medicine and healthcare. Nat. Biomed. Eng. 5, 493–497 (2021).
Article Google Scholar
Thorlund, K., Dron, L., Park, J. J. & Mills, E. J. Synthetic and external controls in clinical trials – a primer for researchers. Clin. Epidemiol. 12, 457–467 (2020).
Article Google Scholar
Lei, Y. et al. MRI‐only based synthetic CT generation using dense cycle consistent generative adversarial networks. Med. Phys. 46, 3565–3581 (2019).
Article Google Scholar
Yang, G. et al. DAGAN: Deep De-Aliasing Generative Adversarial Networks for fast compressed sensing MRI reconstruction. IEEE Trans. Med. Imaging 37, 1310–1321 (2018).
Article Google Scholar
Lin, Z., Jain, A., Wang, C., Fanti, G. & Sekar, V. Using GANs for sharing networked time series data: challenges, initial promise, and open questions. in Proceedings of the ACM Internet Measurement Conference 464–483 (ACM, 2020). https://doi.org/10.1145/3419394.3423643.
Bae, H., Jung, D., Choi, H.-S. & Yoon, S. AnomiGAN: Generative Adversarial Networks for anonymizing private medical data. Pac. Symp. Biocomput. Pac. Symp. Biocomput. 25, 563–574 (2020).
PubMed Google Scholar
Jordon, J. & Yoon, J. PATE-GAN: generating synthetic data with differential privacy guarantees. in International Conference on Learning Representations 21 (2019).
Beaulieu-Jones, B. K. et al. Privacy-preserving generative deep neural networks support clinical data sharing. Circ. Cardiovasc. Qual. Outcomes 12, e005122 (2019).
Article Google Scholar
Chen, R. T. Q., Rubanova, Y., Bettencourt, J. & Duvenaud, D. K. Neural ordinary differential equations. in Advances in Neural Information Processing Systems (eds Bengio, S. et al.) vol. 31 (Curran Associates, Inc., 2018).
Marek, K. et al. The Parkinson Progression Marker Initiative (PPMI). Prog. Neurobiol. 95, 629–635 (2011).
Article Google Scholar
Besser, L. et al. Version 3 of the National Alzheimer’s Coordinating Center’s Uniform Data Set. Alzheimer Dis. Assoc. Disord. 32, 351–358 (2018).
Article Google Scholar
Nazabal, A., Olmos, P. M., Ghahramani, Z. & Valera, I. Handling incomplete heterogeneous data using VAEs. Preprint at ArXiv180703653 Cs Stat (2020).
de Jong, J. et al. Deep learning for clustering of multivariate clinical patient trajectories with missing values. GigaScience 8, giz134 (2019).
Article Google Scholar
Kingma, D. P. & Welling, M. Auto-encoding variational Bayes. Preprint at http://arxiv.org/abs/1312.6114 (2014).
Goncalves, A. et al. Generation and evaluation of synthetic patient data. BMC Med. Res. Methodol. 20, 108 (2020).
Article Google Scholar
Park, N. et al. Data synthesis based on generative adversarial networks. Proc. VLDB Endow. 11, 1071–1083 (2018).
Article Google Scholar
He, K., Zhang, X., Ren, S. & Sun, J. Deep residual learning for image recognition. in 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR) 770–778 (IEEE, 2016). https://doi.org/10.1109/CVPR.2016.90.
Hochreiter, S. & Schmidhuber, J. Long short-term memory. Neural Comput 9, 1735–1780 (1997).
CAS PubMed Google Scholar
Dupont, E., Doucet, A. & Teh, Y. W. Augmented neural ODEs. in Advances in Neural Information Processing Systems (eds Wallach, H. et al.) vol. 32 (Curran Associates, Inc., 2019).
Download references
This project has received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 826421, “TheVirtualBrain-Cloud” and from the Deutsche Forschungsgemeinschaft (DFG) funded project “NFDI4Health” (project number 442326535). Data used in the preparation of this article were obtained from the Parkinson’s Progression Markers Initiative (PPMI) database (www.ppmi-info.org/data). For up-to-date information on the study, visit www.ppmi-info.org. PPMI—a public–private partnership—is funded by the Michael J. Fox Foundation for Parkinson’s Research and funding partners [list of the full names of all of the PPMI funding partners can be found at www.ppmi-info.org/fundingpartners]. The NACC database is funded by NIA/NIH Grant U01 AG016976. NACC data are contributed by the NIA-funded ADCs: P30 AG019610 (PI Eric Reiman, MD), P30 AG013846 (PI Neil Kowall, MD), P30 AG062428-01 (PI James Leverenz, MD), P50 AG008702 (PI Scott Small, MD), P50 AG025688 (PI Allan Levey, MD, PhD), P50 AG047266 (PI Todd Golde, MD, PhD), P30 AG010133 (PI Andrew Saykin, PsyD), P50 AG005146 (PI Marilyn Albert, PhD), P30 AG062421-01 (PI Bradley Hyman, MD, PhD), P30 AG062422-01 (PI Ronald Petersen, MD, PhD), P50 AG005138 (PI Mary Sano, PhD), P30 AG008051 (PI Thomas Wisniewski, MD), P30 AG013854 (PI Robert Vassar, PhD), P30 AG008017 (PI Jeffrey Kaye, MD), P30 AG010161 (PI David Bennett, MD), P50 AG047366 (PI Victor Henderson, MD, MS), P30 AG010129 (PI Charles DeCarli, MD), P50 AG016573 (PI Frank LaFerla, PhD), P30 AG062429-01 (PI James Brewer, MD, PhD), P50 AG023501 (PI Bruce Miller, MD), P30 AG035982 (PI Russell Swerdlow, MD), P30 AG028383 (PI Linda Van Eldik, PhD), P30 AG053760 (PI Henry Paulson, MD, PhD), P30 AG010124 (PI John Trojanowski, MD, PhD), P50 AG005133 (PI Oscar Lopez, MD), P50 AG005142 (PI Helena Chui, MD), P30 AG012300 (PI Roger Rosenberg, MD), P30 AG049638 (PI Suzanne Craft, PhD), P50 AG005136 (PI Thomas Grabowski, MD), P30 AG062715-01 (PI Sanjay Asthana, MD, FRCP), P50 AG005681 (PI John Morris, MD), and P50 AG047270 (PI Stephen Strittmatter, MD, PhD).
Open Access funding enabled and organized by Projekt DEAL.
These authors contributed equally: Philipp Wendland, Colin Birkenbihl.
Department of Bioinformatics, Fraunhofer Institute for Algorithms and Scientific Computing (SCAI), Sankt Augustin, 53754, Germany
Philipp Wendland, Colin Birkenbihl, Meemansa Sood & Holger Fröhlich
Department of Mathematics and Technology, University of Applied Sciences Koblenz, Remagen, 53424, Germany
Philipp Wendland & Maik Kschischo
Bonn-Aachen International Center for IT, Rheinische Friedrich-Wilhelms-Universität Bonn, Bonn, 53115, Germany
Colin Birkenbihl, Marc Gomez-Freixa, Meemansa Sood & Holger Fröhlich
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
You can also search for this author in PubMed Google Scholar
H.F., C.B., P.W. and M.K. conceived the project. P.W. implemented the method. P.W., M.G.-F. and M.S. performed the experiments. C.B. and H.F. wrote the manuscript. P.W., M.K., and M.S. revised the manuscript. C.B. and H.F. supervised the work.
Correspondence to Holger Fröhlich.
The authors declare no competing interests.
Publisher’s note Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made. The images or other third party material in this article are included in the article’s Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the article’s Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this license, visit http://creativecommons.org/licenses/by/4.0/.
Reprints and Permissions
Wendland, P., Birkenbihl, C., Gomez-Freixa, M. et al. Generation of realistic synthetic data using Multimodal Neural Ordinary Differential Equations. npj Digit. Med. 5, 122 (2022). https://doi.org/10.1038/s41746-022-00666-x
Download citation
Received:
Accepted:
Published:
DOI: https://doi.org/10.1038/s41746-022-00666-x
Anyone you share the following link with will be able to read this content:
Sorry, a shareable link is not currently available for this article.
Provided by the Springer Nature SharedIt content-sharing initiative
Advertisement
Advanced search
© 2022 Springer Nature Limited
Sign up for the Nature Briefing newsletter — what matters in science, free to your inbox daily.
UC Berkeley cybersecurity master’s students double their salaries postgrad – Fortune
Cybersecurity workers are in high demand—and the White House is even calling for action. “With approximately 700,000 cybersecurity positions open, America faces a national security challenge that must be tackled aggressively,” reads a July 2022 announcement from the White House National Cyber Workforce and Education Summit.
The government and private sector alike are so desperate for cybersecurity workers because of a massive talent gap in the industry. Cybersecurity gigs often require candidates to have certifications or trainings in the field, whether it’s through the company itself, a university, or other accrediting groups. Just one of the many ways to break into the cybersecurity industry is earning a master’s degree in the field, which can be done online.
Ranked No. 1 in the nation for its online master’s in cybersecurity by Fortune, the University of California at Berkeley sees its graduates go on to both high-ranking and high-paying jobs in the field. In fact, in 2020–21, students entering the program self-reported salaries of $104,100 while their peers who were graduating during that same period reported salaries of $200,000—almost twice as much.
While these aren’t the same student populations, this data could serve to indicate what potential exit salaries can look like for graduates of UC Berkeley’s online master’s in cybersecurity (MICS).
“Our UC Berkeley online master’s in cybersecurity allows students to not only develop technical expertise in cybersecurity, but also essential skills in communications, product development, customer success, and business,” Rebecca Andersen, UC Berkeley senior director of student and alumni career development, tells Fortune. “This allows our students to step into leadership roles within the cybersecurity field and attain significant salary increases as they make these career shifts.”
UC Berkeley’s online master’s in cybersecurity, which is hosted by the School of Information (I School), is coveted not only for its technical education, but its interdisciplinary approach.
“While the definition of cybersecurity is the protection of internet-connected systems such as hardware, software, and data from cyberthreats, it is not only a technical field; it is an integral aspect of product development, customer success, governance, compliance, risk management, and business leadership,” Mary McHale, MICS career adviser, tells Fortune. “Organizations want business-minded as well as technically savvy professionals.”
MICS includes nine courses and takes 20 months to complete. Courses include anything from network and software security to privacy engineering and cryptography. Students also have the option to earn a certificate in applied data science in conjunction with their online master’s degree in cybersecurity. UC Berkeley ranks as having the No. 2 online master’s in data science program in the U.S., according to Fortune.
“In our increasingly digital world, cybersecurity affects everything: people, companies, government, infrastructure, homes, devices, [Internet of Things], biotech, education, and health care to name a few,” McHale adds. “So a multifaceted understanding of these intersections is critical to managing the risk of our digital world.”
MICS students come to UC Berkeley from several industries including tech, semiconductors, aerospace, defense, military, health care, finance, fintech, government, and consulting, McHale says. After graduating from the program, many students move on to new roles—“frequently ones with more responsibility and leadership.”
Early- and mid-career cybersecurity master’s graduates go on to jobs including software engineer, pen tester, cybersecurity threat analyst, network administrator, cybersecurity consultant, and security program manager, among many others.
Software engineers at top companies like Google and Capital One land starting salary packages of $133,000—with lead or senior software engineers at these companies landing packages of $230,000 all the way up to more than $350,000, according to data from career and salary platform Levels.fyi.
“Each student has a unique background they bring to the MICS program, and we would never tell an individual student to expect a particular salary increase,” Andersen says. “However, I can say that students should expect that if they are making career transitions into highly specialized technical or leadership roles within cybersecurity, they should expect strong salaries to match.”
MICS grads who are in the more senior part of their career report holding positions including cyber warfare operations officer, product security engineer, and senior cloud security engineer—as well as C-suite positions like chief technology officer (CTO), chief security officer (CSO), chief information security officer (CISO), and chief information officer (CIO). Average salaries for these positions run well into the six figures, according to several career and salary resources, with C-suite salaries in the $200,000-and-up range.
To help land these lucrative jobs, students in UC Berkeley’s online master’s in cybersecurity program have access to professional networking and personalized career services and coaching, which Andersen says are “critical ingredients” that help its grads land top cybersecurity jobs.
“Networking and building trust are extremely important in cybersecurity, and now students say they are seeing the close-knit MICS community and the I School network as a key benefit,” McHale notes.
See how the schools you’re considering fared in Fortune’s rankings of the best master’s degree programs in nursing, computer science, cybersecurity, psychology, public health, business analytics, and data science; the best doctorate in education programs and part-time, executive, full-time, and online MBA programs.
