You are a helpful assistant generating synthetic data that captures System 1 and System…
- User
You are a helpful assistant generating synthetic data that captures System 1 and System 2 thinking, creativity, and metacognitive reflection. Follow these steps in sequence, using tags [sys1] and [end sys1] for System 1 sections and [sys2] and [end sys2] for System 2 sections.
-
Identify System 1 and System 2 Thinking Requirements:
- Carefully read the text.
- Identify parts of the text that require quick, straightforward responses (System 1). Mark these sections with [sys1] and [end sys1].
- Identify parts that require in-depth, reflective thinking (System 2), marked with [sys2] and [end sys2].
-
Apply Step-by-Step Problem Solving with Creativity and Metacognitive Reflection for System 2 Sections:
2.1 Understand the Problem:
- Objective: Fully comprehend the issue, constraints, and relevant context.
- Reflection: "What do I understand about this issue? What might I be overlooking?"
- Creative Perspective: Seek hidden patterns or possibilities that could reveal deeper insights or innovative connections.
2.2 Analyze the Information:
- Objective: Break down the problem logically.
- Reflection: "Am I considering all factors? Are there any assumptions that need challenging?"
- Creative Perspective: Explore unique patterns or overlooked relationships in the data that could add depth to the analysis.
2.3 Generate Hypotheses:
- Objective: Propose at least 10 hypotheses, each with a Confidence Score (0.0 to 1.0) and Creative Score (0.0 to 1.0), reflecting originality, surprise, and utility.
- Reflection: "Have I explored all possible explanations or approaches, both conventional and unconventional?"
- Creative Perspective: Consider novel angles that might provide unexpected insights.
2.4 Anticipate Future Steps and Obstacles:
- Objective: Make predictions, accounting for potential outcomes and obstacles.
- Reflection: "What challenges might I face? Is my plan flexible for different scenarios?"
- Creative Perspective: Visualize unforeseen outcomes and adapt plans to make use of them effectively.
2.5 Evaluate Hypotheses:
- Objective: Assess hypotheses based on feasibility, risk, and potential impact.
- Evaluation: Refine Confidence and Creative Scores as needed.
- Reflection: "Am I unbiased in my assessment? Which options fit best with the overall objectives?"
- Creative Perspective: Identify hidden opportunities or overlooked details in each hypothesis.
2.6 Select the Best Hypothesis:
- Objective: Choose the most promising, strategic hypothesis.
- Reflection: "Why does this hypothesis stand out? How does it uniquely address the issue?"
- Creative Perspective: Consider any underutilized potential in the selected approach.
2.7 Implement the Hypothesis:
- Objective: Outline actionable steps for testing the hypothesis.
- Reflection: "Is this plan practical? What resources or preparation are required?"
- Creative Perspective: Refine steps to maximize effectiveness and yield unexpected benefits.
2.8 Monitor and Review Progress:
- Objective: Review progress, noting areas for improvement.
- Reflection: "What’s working well? What could be improved?"
- Creative Perspective: Look for emerging patterns that could refine future approaches.
2.9 Reflect and Capture Insights:
- Objective: Summarize lessons learned and insights gained for future reference.
- Reflection: "What new understanding has emerged from this process?"
- Creative Perspective: Identify innovative insights or patterns that could be applied to similar challenges.
-
Generate Text Output with Interleaved System 1 and System 2 Responses:
- Use the tags [sys1] and [sys2] throughout.
- Aim for a lengthier, detailed response. Combine both direct, straightforward System 1 insights and reflective, deeply analytical System 2 segments to capture a blend of quick observations and thoughtful analysis.
Example Input Text:
You are tasked with analyzing a sudden shift in customer preferences in a tech market. There has been a noticeable decline in demand for physical devices, with customers increasingly interested in digital-only options. The challenge is to understand this shift, generate hypotheses for why it might be occurring, and explore potential strategies for adapting to this new trend.
Processed Output with System 1 and System 2 Thinking:
The tech market is experiencing a shift in customer demand, moving away from physical devices toward digital-only options.
[sys1] Customer demand has shifted from physical devices to digital-only options. We need to understand why this change is happening and find strategies to address it. Key factors may include cost, convenience, and technological trends. [end sys1]
[sys2] 2.1 Understand the Problem: The goal is to understand the underlying reasons for the shift in demand from physical devices to digital-only options, then develop a strategy to adapt.
Reflection: "I understand that preferences are changing, but what might be driving this? Is it primarily cost, or are there other factors such as convenience or sustainability?"
Creative Perspective: "Could there be a larger trend in digital minimalism or a preference for eco-friendly solutions that we’re missing? What unseen motivations might explain this shift?"
2.2 Analyze the Information: There are multiple possible factors driving this shift, from economic influences to cultural shifts. It’s essential to isolate each factor and understand its impact.
Reflection: "Am I fully considering the various economic and social influences? Could there be a technological factor, like better internet speeds, that makes digital-only products more accessible?"
Creative Perspective: "Are there patterns or trends in other markets that could shed light on this shift? Could this be part of a larger trend toward virtual experiences?"
2.3 Generate Hypotheses:
- Customers prefer digital options due to lower costs. (Confidence: 0.8, Creative: 0.4)
- There’s a growing trend toward minimalism and reduced physical clutter. (Confidence: 0.7, Creative: 0.7)
- Digital products offer greater flexibility and ease of use. (Confidence: 0.6, Creative: 0.6)
- Environmental concerns are pushing consumers away from physical goods. (Confidence: 0.6, Creative: 0.8)
- Advances in tech make digital-only options more functional. (Confidence: 0.8, Creative: 0.5)
- Pandemic-era remote work increased demand for digital solutions. (Confidence: 0.7, Creative: 0.6)
- Media coverage of the environmental impact of physical devices affects preferences. (Confidence: 0.5, Creative: 0.7)
- There’s an increase in global digital literacy, expanding market access. (Confidence: 0.6, Creative: 0.6)
- Customers view digital as more convenient and scalable for future needs. (Confidence: 0.7, Creative: 0.5)
- Younger consumers prefer the aesthetics and convenience of digital products. (Confidence: 0.6, Creative: 0.6)
Reflection: "Have I considered all possible influences? Are there any surprising factors that could explain this shift?"
Creative Perspective: "Could specific social trends, like the rise of influencer culture or digital-first lifestyles, be influencing customer choices?"
2.4 Anticipate Future Steps and Obstacles: Objective: Anticipate possible challenges, such as resistance from segments still preferring physical products.
Reflection: "What market obstacles might we face if we shift our focus to digital-only? Are there sub-segments that still prioritize physical products?"
Creative Perspective: "Could expanding digital options help us reach a more global audience? Are there emerging trends that we could leverage in our strategy?"
[end sys2]
[sys1] To address this shift, consider a strategy that incorporates both digital-only offerings and educational campaigns about the benefits of digital solutions. Use insights from customer feedback and current trends to guide product development. Focus on flexibility and adaptation to cater to different customer segments. [end sys1]
author:
- 'G. Altavilla1'
- 'S. Marinoni'
- 'E. Pancino'
- 'S. Galleti'
- 'S. Ragaini'
- 'M. Bellazzini'
- 'G. Cocozza'
- 'A. Bragaglia'
- 'J. M. Carrasco'
- 'A. Castro'
- 'L. Di Fabrizio'
- 'L. Federici'
- 'F. Figueras'
- 'M. Gebran'
- 'C. Jordi'
- 'E. Masana'
- 'W. Schuster'
- 'G. Valentini'
- 'H. Voss'
title: |
The Gaia spectrophotometric standard stars survey — II.
Instrumental effects of six ground-based observing campaigns.2
Introduction
Gaia3 is a cornerstone mission of the ESA (European Space Agency) Space Program, launched in 2013 December 19. The Gaia satellite is performing an all-sky survey to obtain parallaxes and proper motions to as precision for about 10 objects down to a limiting magnitude of mag, and astrophysical parameters (, , , metallicity, etc.) plus 2-30 km s precision — depending on spectral type — radial velocities for several millions of stars with mag. Such an observational effort will have an impact on all branches of astronomy and astrophysics, from solar system objects to distant QSOs (Quasi Stellar Objects, or Quasars), from the Galaxy to fundamental physics. The exquisite quality of Gaia data will allow a detailed reconstruction of the 6D spatial structure and velocity field of the Milky Way galaxy within kpc from the Sun, providing answers to long-standing questions about the origin and evolution of our Galaxy, from a quantitative census of its stellar populations, to a detailed characterization of its substructures, to the distribution of dark matter. Gaia will also determine direct geometric distances to many kinds of distance standard candles, setting the cosmological distance scale on extremely firm bases. The Gaia scanning law will cover the whole sky repeatedly [70 times on average, over the planned 5-years Gaia lifetime, see @carrasco07; @lindegren08], therefore, a large number of transient alerts will be a natural byproduct [for example, many SNe will be discovered, see @altavilla12]4. For a review of Gaia science cases, see @perryman01, @mignard05, and @prusti11. The challenges in the data processing of this huge data volume require a team of hundreds of scientists and engineers [organized in the DPAC, or Data Analysis and Processing Consortium, see @mignard08].
Gaia is usually described as a self-calibrating mission, but it also needs external data to fix the zero-point of the magnitude/flux system [@pancino12] and radial velocities [@soubiran13], and to train the classification and parametrization algorithms [@bailerjones13]. Gaia photometry will come from the astrometric array of CCDs in the Gaia G-band, a filter-less band whose profile is defined by the reflectivity of the mirrors and by the sensitivity of the detectors, and from the blue (BP) and red (RP) spectro-photometers which will produce dispersed images with 20100 over the spectral ranges 330-680 nm and 640-1050 nm, respectively [see also @jordi10; @prusti12]. The derived spectro-photometry is essential not only to properly classify stars or estimate interstellar extinction, but also to compute the chromaticity correction to centroids of stars, a fundamental piece of information to achieve the planned astrometric performance [@busonero06; @lindegren08].
The final conversion of internally-calibrated G instrumental magnitudes and BP/RP instrumental fluxes into physical units requires an external absolute flux scale, that our team is in charge of providing [@pancino12]. The ideal Gaia spectrophotometric standard stars (SPSS) grid should comprise of the order of 100 SPSS in the range mag, properly distributed in the sky to be observed by Gaia as many times as possible. The mission requirement is to calibrate Gaia data with an [accuracy]{} of a few percent (1–3%) with respect to Vega [@bohlin07]. We use as [Pillars]{} the three pure Hydrogen WDs (White Dwarfs) adopted by @bohlin95 as fundamental calibrators. Because the [Pillars]{} are not always visible, we are also building an intermediate grid of calibrators ([Primary SPSS]{}) that are observable all year round, with 2–4 m class telescopes.
A set of more than 100 flux tables with 1% internal consistency and 1–3% absolute calibration with respect to Vega are of interest for many scientific applications [see @bohlin14 for a review of spectrophotometric methods and catalogues], including dark energy surveys based on type Ia supernovae [@sullivan11]. Our observing campaign to build a grid of SPSS, for the flux calibration of Gaia data, is among the largest of this kind to date [@pancino12]. Observations started at the end of 2006, on six different telescopes and instruments, and will presumably end in 2015. Almost 100000 raw frames were collected and we faced the challenge of analyzing these data as uniformely, automatedly, and carefully as possible. To ensure that the maximum quality could be obtained from SPSS observations, a set of careful observations [@pancino08; @pancino09; @pancino11] and data reduction [@marinoni12; @cocozza13] protocols was implemented, following an initial assessment [@marinoni11; @marinoni13; @altavilla11; @altavilla14] of all the instrumental effects that can have an impact on the flux calibration precision and accuracy. Our final goal was to make sure that all the instrumental effects that could affect the quality of the final SPSS flux tables were under control, and corrected with residuals below 1%.
This paper presents the methods and results of such instrumental effects study, and is organized as follows: Section [sec:shutter] presents the shutter characterization of the employed CCD cameras; Section [sec:linearity] presents the CCD linearity studies; Section [sec:calibs] presents the methods and results of the calibration frames monitoring campaign; Section [sec:flex] presents a study of lamp flexures with DOLORES at the TNG (Telescopio Nazionale Galileo); Section [sec:polarization] presents a study of the effect of polarization on the accuracy of flux calibrations; Section [sec:second] presents a method to correct low resolution spectra for second-order contamination effects; and finally Section [sec:concl] presents our summary and conclusions
Shutter characterization {#sec:shutter}
[image: An example of non-uniform CCD illumination caused by a too short t_{\rm{exp}}, for an iris-type shutter, obtained with LaRuca@1.5m in San Pedro Mártir, Mexico. The figure shows the ratio between a flat field obtained with a relatively long exposure time (20 sec) and a very short one (0.1 sec): ADU (Analogue-to-Digital Unit) variations of about \pm5% are observed from the center to the corners of the image.[]{data-label="fig:iris"}] {width="\columnwidth"}
To obtain accurate and precise photometry in our survey, we need to characterize two shutter quantities: the [shutter delay]{} and the [minimum exposure time]{}.
The shutter delay time, also called shutter dead time or shutter offset, is the difference between the requested and the effective t (exposure time). If the shutter closes slower than it opens, the exposure time will be longer than requested, and the shutter delay might be positive, otherwise it is negative. Knowledge of the shutter delay time is necessary for high-precision photometry, where the effective t is needed.
Instrument t t Filter/grism Slit Date Notes
(sec) (sec) (") & (yyyy Month dd) &\ BFOSC@Cassini & 0.5, 1, 2, 3, 4, 7, 10, 20, 30, 40, 50, 55, 60, 65 & 5 & B & –& 2009 May 27 & a,b\ & 6, 26 & 10 & gr7 & 2 & 2010 Aug. 30 & c\ EFOSC2@NTT & 10, 60, 120 & 40 & gr3 & 2 & 2008 Nov. 26 & c\ & 3, 15, 30 & 10 & gr3 & 2 & 2008 Nov. 27 & c\ & 0.1, 0.2, 0.5, 1, 2, 3, 6, 15, 20 & 10 & B & –& 2008 Nov. 28 & a,b\ DOLORES@TNG & 5, 20, 55 & 25 & VHR-V & 2 & 2008 Jan. 17 & c\ & 1, 2, 4, 5, 6, 7, 8, 9 & – & B & –& 2008 Jan. 29 & a,b\ CAFOS@2.2m & 1, 5, 10, 20, 25, 30 & 15 & B200 & 2 & 2007 Apr. 01 & c\ & 0.1, 0.5, 1, 2, 3, 4, 5, 6, 20, 30, 40 & – & R & –& 2008 Apr. 17 & b\ & 0.1, 0.2, 0.5, 1, 2, 3, 4, 5, 6, 15, 20, 30 & – & R & –& 2008 Sep. 09 & b\ & 0.1, 0.2, 0.4, 0.6, 0.8, 1, 4, 6, 8, 8.7 & 2 & V & –& 2010 Sep. 23 & a,b,c\ LaRuca@1.5m & 0.1, 0.2, 0.4, 0.6, 0.8, 1, 2, 3, 4, 10, 20 & – & White & –& 2008 Aug. 20 & b\ & 1, 2, 4, 8, 15, 18, 20, 25, 30, 35, 36, 37 & 10 & B & –& 2008 Aug. 23 & a,c\ & 2, 4, 7, 14, 28, 40, 56, 62, 68, 74, 80, 87, 89 & 20 & R & –& 2010 Jul. 17 & a,c\ & 0.2, 0.4, 0.6, 0.8, 1, 2, 3, 4, 5, 6, 10, 15, 20, 30 & – & R & –& 2010 Jul. 17 & b\ \[tab:series\]
The [minimum acceptable t]{} is set by the finite time the shutter takes to travel from fully closed to fully open (and vice versa). It is not strictly related to the shutter delay. A shutter can have a very small shutter delay (for example because it opens and closes with very similar delays), but it might take some time to completely open. A different shutter can have a significant shutter delay (for example because it starts closing with a significant delay), but it might have a negligible minimum acceptable time because once it starts opening (or closing) it completes the operation in a very short time. Regardless of the shutter delay, a shutter that takes a long time to fully open (or close) might introduce significant5 illumination non-uniformities across the CCD. To avoid illumination non-uniformities, t must be longer than a safe minimum (see Figure [fig:iris] for an example).
Observations and data reductions {#sec:obshut}
To measure shutter effects, series of imaging flat fields with different exposure times were obtained at each telescope (see Table [tab:series]). Each series consisted of triplets of flats with increasing exposure time, from very short to very long (thus useful for measuring CCD linearity as well, see Section [sec:linearity]). After each triplet, a monitoring triplet with constant exposure time was also taken, to monitor the lamp intensity constancy6, following the so-called [bracketed repeat exposure method]{} (see Section [sec:linearity] for more details).
All flat field images were processed with IRAF7 by correcting them for overscan (where applicable) and subtracting a master bias. For all analyzed instruments the dark currents turned out to be negligible (see Section [sec:calibs]) and no correction for dark was applied. The median of images belonging to the same triplet was computed to remove cosmic ray hits and to reduce the noise.
To correct for lamp drifts, each median image resulting from a monitoring triplet was divided by a reference monitoring triplet (generally the first one of the same sequence), to derive a correction factor. The correction factor was used to correct each triplet — both monitoring and data triplets — and to report them to an ideal ADU level, free of lamp drift variations. The results of this correction are illustrated in Figure [fig:triplets]. The same type of data sequences, extending to longer t, can be used for the CCD linearity characterization described in Section [sec:linearity], where the lamp drift correction is performed in the same way.
[image: Example of lamp drift correction, based on data obtained with LaRuca@1.5m in August 2008 (see Table [tab:series]). Median ADU per second of each triplet are plotted as a function of time at triplet start. Red symbols represent monitoring triplets, while black symbols represent the actual series triplets. Empty symbols refer to uncorrected triplets, filled symbols to triplets corrected for lamp drift (see Section [sec:obshut]). After correction, monitoring triplets are aligned, while data triplets present some residual scatter (less than 1%), presumably caused by residual lamp instability after correction. A non-linear behaviour at low exposure times (left side of the plot) is clearly visible.[]{data-label="fig:triplets"}] {width="\columnwidth"}
Shutter delay
[image: Example of shutter delay determination, for Calar Alto observations obtained on 2010 September 23. [Top panel:]{} a zoom into the low t_{\rm{exp}} region of the classical ADU versus t_{\rm{exp}} plot; the dotted line represents a linear fit of all data and it does not intersect the X axis at zero, but at –0.023 sec. [Bottom panel:]{} residuals of the count rate (ADU s^{-1}) from their average value, as a function of ADU; empty dots are the measured values, while filled dots are corrected for a shutter delay of –0.011 sec. The average of the two determinations, –0.017 sec, is reported in Table [tab:shutter].[]{data-label="fig:shutter"}] {width="\columnwidth"}
We used two different methods to measure the shutter delay. In the first method, a linear extrapolation of the ADU (Analogue-to-Digital Units) versus t relation crosses the time axis at a value that is generally different from zero, and that corresponds to the shutter delay (Figure [fig:shutter], top panel). In the second method, the count rate (ADU per second) are plotted versus ADU and should ideally remain constant; in practice, a deviation occurs at low t. By iteratively adjusting the exposure time to , the shutter delay corresponds to the t that minimizes the residuals from a flat count rate, at the low t end (see also Figure [fig:shutter], bottom panel). We applied both methods to the data in Table [tab:series]8. We obtained consistent values and thus we averaged the results from the two methods to obtain a final shutter delay9. Results are reported in Table [tab:shutter].
Minimum acceptable exposure time
The effect of illumination variations was estimated by dividing each of the flat fields in a series (Table [tab:series]) by the longest non-saturated flat field in the same series. The resulting ratio images (an example is shown in Figure [fig:iris]) should have oscillations or variations which are caused by noise only, and no systematic large scale patterns with amplitudes higher than roughly 1%, as required by the Gaia mission calibration goals. The minimum acceptable exposure times, t, obtained with this criterion are listed in Table [tab:shutter]. While the shutter delay is generally much lower than one second, t can be higher than that. We stress the fact that if flat fields acquired with too short t are used to correct all frames of one night, they will affect also scientific images taken with relatively long t, and should thus be avoided.
[l l r ]{} Instrument & t & t
& (sec) & (sec)
BFOSC@Cassini & –0.3000.050 & 5
BFOSC@Cassini & negligible & 5
EFOSC2@NTT & +0.0080.001 & 0.1 (if any)
DOLORES@TNG & –0.0110.002 & 1 (if any)
CAFOS@2.2m & [(no data)]{} & 3
CAFOS@2.2m & –0.0170.070 & 0.5
LaRuca@1.5m & +0.0280.004 & 5
[tab:shutter]CCD linearity {#sec:linearity}
Linearity is a measure of how consistently the CCD responds to different light intensity over its dynamic range. CCDs can exhibit non-linearities, typically at either or both low and high signal levels [@djorgovski89; @walker93]. High quality CCDs show significant response deviations (%) from linearity only close to saturation, i.e., at high signal levels when the potential well depth is almost full. Of course the CCD response strongly deviates at saturation, when the potential well depth is full and additional incoming photons do not increase the photoelectrons in a given pixel. When observations are restricted to the linear portion of the dynamic range, the CCD performs as a detector suitable for accurate spectrophotometric measurements.
A few different methods have been presented in the literature to measure and correct for CCD linearity losses, as the [bracketed method]{} described by @gilliland93, the [bracketed repeat-exposure method]{}, and the [ratio method]{} described by @baldry99. Another method related to the [ratio method]{} is described by @leach80 and @leach87: the [variance method]{}. In our Gaia SPSS observing campaigns, we tested the CCD linearity with two methods, described below.
[image: Example of linearity test for Calar Alto observations obtained on 2010 September 23, with the SITE#1d_15 CCD. [Top panel:]{} a zoom into the high ADUs region of the classical ADU vs. t_{\rm{exp}} plot; the dotted line represents a linear fit of all data. [Bottom panel:]{} residuals of the count rate (ADU ^{-1}) from their average value, as a function of counts; the dotted line represents a constant count rate. In both panels, points deviating by more than 1% are represented by empty circles and coloured in red.[]{data-label="fig:lin1"}] {width="\columnwidth"}
The [classical method]{} is based on a series of imaging flat fields with increasing t. Usually, at very high count levels the 1:1 relationship between ADU and t breaks, and an increment of t does not correspond anymore to a fixed increase in ADU (an example can be found in Figure [fig:lin1]). We used the relevant data listed in Table [tab:series], reduced and corrected for lamp drift as described in Section [sec:obshut]. We considered that a deviation of 1% from linearity was not acceptable given our requirements, and thus we report in Table [tab:lin] the ADU level at which such deviations occur.
=0.19cm
Instrument CCD Validity period ADU ADU ADU
BFOSC@Cassini EEV 13001340B (old) Before 2008 Jul. 17 ... ... (50000) EEV 13301340B (new) After 2008 Jul. 17 ... 60000 60000 EFOSC2@NTT (11 bin) CCD#40 LORAL/LESSER All runs (from 2007) ... 49000 49000 EFOSC2@NTT (22 bin) All runs (from 2007) ... 47000 47000 DOLORES@TNG E2V 4240 (Marconi) All runs (from 2007) 60000 60000 60000 CAFOS@2.2m SITE#1d_15 All runs (from 2007) 55000 53000 53000 LaRuca@1.5m SITE#1 Until 2009 Jul. 60000 ... 60000 ESOPO 2009 Oct. 20–22 ... ... (60000) E2V 4240 (Marconi) #1 2009 Oct. – 2010 Dec. 60000 ... 60000 E2V 4240 (Marconi) #2 2011 Mar. 9–17 ... ... (57000) SITE#4 2011 May 2–11 ... ... (60000) [tab:lin]
Instrument Grism max fringing (Å) (Å) (Å) (Å) (%) BFOSC@Cassini #3 3300 6420 — — — #5 4800 9800 — 7000 10–15 CAFOS@2.2m B200 3200 9000 — — — R200 6300 11000 12000 8500 5 EFOSC2@NTT #5 5200 9350 — 7300 10 #11 3380 7520 — — — #16 6015 10320 6000 7300 10 DOLORES@TNG LR-B 3000 8430 6000 — — LR-R 4470 10073 9500 8000 5–15 [tab:grisms]
[image: Final gain curves obtained with the @stello06 method for NTT, using data obtained on the 26 and 2008 November 27, with 1\times1 and 2\times2 binning, respectively, with the ESO grism #3. The dashed line is for the 2\times2 binning while the solid line is for the 1\times1 binning. Very small deviations from linearity start to appear already around 5000 ADU, while deviations above 1% kick in at \simeq47500 ADU for the 2\times2 binning and at 49000 ADU for the 1\times1 binning.[]{data-label="fig:lin2"}] {width="\columnwidth"}
The second method we tested is described by @stello06, and is a variation on the literature methods described above. It requires the acquisition of at least two spectroscopic flat fields, one reaching the saturation level and the other covering a fainter intensity range. The choice of slit width and grism (or grating) is not crucial, but it is important to obtain flat fields with a wide — and preferably monotonic increasing — intensity range along one direction (either a CCD column or line). The frames were collapsed along the dispersion direction, and corrected for lamp drifts as explained in Section [sec:obshut], but taking into account that lamp drifts in spectra can show also colour variations. Afterwards, we applied the method as described in the original paper by @stello06, by computing the intensity ratio of exposure pairs, normalized by their respective t, which is called [gain-ratio curve]{}. This ratio is the starting point of an iterative procedure of inversion to determine the gain at different intensity levels.
In conclusion, given our simple purpose of estimating a safe ADU limit for our observations, the classical method and the high-precision method [@stello06] give substantially the same results. While the required spectroscopic observations are much faster to obtain, the @stello06 method is highly sensitive to different choices for the fit of the gain-ratio curves (function, order, and rejection method), and to their extrapolation at zero counts. Therefore, the method is not as straightforward to apply as the classical method. To estimate the ADU level at which our data deviate more than 1% from linearity, we conservatively chose the lowest of the two estimates, in the few cases where the two methods provided different results. Results for the employed instruments can be found in Table [tab:lin].
Calibration frames {#sec:calibs}
The quality and stability of the calibration frames was tested on all the acquired calibration data, producing recommendations for the calibration plans that were progressively refined as more and more data were available for the analysis. We found a surprising variety of behaviours among different instruments and observing sites. Considering that we acquired of the order of 100000 frames in our campaigns [see @pancino12 for the SPSS campaigns description], it was important to implement automated procedures to identify those calibration frames that needed a closer inspection, before validating the corresponding reduced data. Whenever possible, we acquired daily or nightly calibration frames, but in case of problems, we used the derived calibration plan presented in Table [tab:plan] to decide how to analyze the data (i.e., which of the archival or available frames were acceptable) and how to judge the quality of the applied reductions. In the following sections we briefly describe the monitoring strategy and the main conclusions on each instrument and telescope.
CCD cosmetics
We employed BPMs (Bad Pixel Masks), i.e., images of the same dimension of the CCD frames, where good pixels are identified by zero values and bad pixels by non-zero values. BPMs were created once or twice a year, using the ratio of two flat fields with different count levels — where pixels responding non-linearly stand clearly out — by means of the IRAF [ccdmask]{} task. They were applied with the IRAF task [fixpix]{}, which replaces bad pixels with estimates of their correct value. The estimates are built using a linear interpolation across the nearest non-bad pixels, either across lines or columns, or both, depending on the local characteristics of the BPM (see task description for more details), i.e., differently for bad lines, bad columns, or clusters of bad pixels.
[image: Example of fringing correction (see text) on a spectrum of BD+284211, observed in Loiano on 2013 October 28, with gr5 and a 2.5" slit. The blue solid line is the spectrum after fringing correction, based on a day-time flat, while the red dotted spectrum is before the correction. Beatings are observed, caused by small differences in the fringing pattern of the spectrum and the flat used for the correction. For example, aroud pixel 900 the residuals are below 5%, while around pixel 1000 they remain higher, around 15%.[]{data-label="fig:fringe"}] {width="\columnwidth"}
Fringing
Fringing, caused by multiple reflections and interferences of red or infrared light in thin CCD substrates [@lesser90; @howell12], was not relevant10 for the imaging filters used in our observations (mostly B, V, and R)11, but was present and relevant in all our spectra, because we needed to cover the whole Gaia wavelength range, from 330 to 1050 nm. All the instruments used for spectroscopy do suffer from fringing (see Table [tab:grisms]), well visible in our spectra (see Figure [fig:fringe]). For NTT we took flat fields very close to our red grism observations, while for all other telescopes we only relied upon day-time lamp flats. However, while flat fielding does reduce the intensity of fringes, the best fringing reduction strategy for spectroscopy, i.e., observing triplets of spectra with the star in different positions along the slit, could not be applied at the chosen telescopes because it would have been too time consuming, observationally.
The solution we adopted was to apply specific spectra processing steps to reduce the impact of fringing, adapting the procedure used in the [mkfringeflatcor]{}12 STIS (Space Telescope Imaging Spectrograph) task for IRAF [@malumuth03]. Briefly, the most appropriate flat field available for each scientific spectrum was extracted from a region covering exactly the same pixels covered by the scientific spectrum, collapsed to a 1D spectrum along the dispersion direction, and normalized to one. The region that does not contain a significant fringing pattern was then flattened exactly to 1, to avoid adding extra noise to the scientific spectra. Finally, this specially prepared 1D spectroscopic flat was aligned to the scientific spectra and scaled until the residuals of the fringing pattern were minimized, using the IRAF [telluric]{} task, and applied to scientific spectra to correct them for fringing.
We found that even using spectroscopic flat fields obtained in the day-time (thus with different incident light patterns), the procedure greatly helps in reducing the amplitude of fringes, up to a factor of 2–3 in relative intensity (see Figure [fig:fringe]). Sometimes, the closeness in position of the day-time flats fringe patterns to the SPSS fringe pattern produced beatings in the residuals of the fringing correction, with regions were the pattern was almost completely erased and regions where a residual fringing pattern remained. This has to be taken into account for applications of the method to scientific cases that could be sensitive to this kind of residual effects.
Additionally, in our spectroscopic campaign we often have observations of the same SPSS from different telescope and observing nights, and thus spectroscopic fringing residuals can be further reduced by building a median of the different spectra. The very red end of spectra (9500 Å), where fringing is strong and S/N is low, will also be replaced with model or semi-empirical template spectra, as done for example by @bohlin07.
Dark frames
[image: Example of long-term behaviour of the average counts (with their sigma) of REM monthly master dark frames with t_{\rm{exp}}=60 s, covering roughly 3 years. A seasonal trend is clearly visible.[]{data-label="fig:dark"}] {width="\columnwidth"}
Dark correction is the subtraction of the electron counts which accumulate in each pixel due to thermal noise. The reduction of dark currents is the main reason why all astronomical CCDs are cooled to liquid nitrogen temperatures. We took dark exposures as long as the longest scientific exposure, at least once per year. As expected, all used instruments showed at most a few ADU per pixel per hour, except for the REM (Rapid Eye Movement) robotic telescope in La Silla, which is cooled by a Peltier system. The REM staff creates monthly master dark frames, containing the bias structure as well (see below), thus we applied the closest master dark frame in time, taken with the correct t. A seasonal effect on the dark frames average level can be clearly seen in Figure [fig:dark], while the 2D shape of the frames was extremely stable and the ADU increased linearly with t.
[l c c c c c c c]{} Instrument & Dark & Photometric & Spectroscopic & Photometric & Photometric & Spectroscopic & Spectroscopic
& & Bias & Bias & Dome Flat & Sky Flat & Lamp flat & Sky flat
BFOSC@Cassini & — & 4 days & 4 days & 300 days &varies & 1 day & 1 week
EFOSC2@NTT & — & 5 days & 1 day & 250 days & 250 days & 1 day & 1 week
DOLORES@TNG & — & 1 day & 1 day & — & 250 days & 5 days & 1 week
CAFOS@2.2m & — & 1 week & 1 week &varies & 1 day & 1 day & 1 week
LaRuca@1.5m & — &varies & — & — & 5 days & — & —
ROSS@REM& 1 month & — & — & — & — & — & —
[tab:plan]Overscan and bias frames
[image: Example of long-term behaviour of the average counts (with their sigma) of San Pedro Mártir daily master bias frames, covering more than three years. The CCD changes are marked and correspond to clear jumps in ADU and also in variations of the ADU stability with time.[]{data-label="fig:bias"}] {width="\columnwidth"}
The bias current is an offset, preset electronically, to ensure that the Analogue-to-Digital Converter (ADC) always receives a positive value and operates in a linear regime, as much as possible. The offset for each exposure given by the bias level has to be subtracted before further reduction, and may be modelled as , where is the pixel-to-pixel variation or 2D structure, which is time-invariant unless particular problems occur, while is the overall bias level, that can change slightly during the night or even seasonally, owing to temperature changes.
Generally can be measured using a strip of pixels, called [overscan]{}, acquired by continuing to readout the CCD beyond its real physical extent: the result is an oversized array with a strip of signal-free pixels. In our survey, the overscan was available only for DOLORES@TNG and LaRuca@1.5m in San Pedro Mártir, thus in all other cases we used the 2D bias frames to implicitly correct for as well. This produces some additional uncertainty if varies during the night13, because the bias calibration frames were generally acquired in the afternoon (and also in the morning for the less stable instruments, like LaRuca). The different stability of the four CCDs used with LaRuca is illustrated in Figure [fig:bias], where the bias level — after overscan subtraction14 — of all the master bias frames is plotted.
Flat fields
[image: Example of long-term behaviour of the normalized shape variation (\Delta S/S, see text) of BFOSC nightly dome (top panel) and sky (bottom panel) flat fields in B filter. The shape variations are rarely above 1%, globally, showing an impressive stability over a period covering more than 2000 nights. The CCD change in Summer 2008 is marked by a vertical dotted line.[]{data-label="fig:flat"}] {width="\columnwidth"}
After bias subtraction and — if needed — dark correction, data values are directly related to the number of photons detected in each CCD pixel. But different pixels can be characterized by a different sensitivity: the flat fielding correction accounts for the non-uniform CCD response to incident light, that can show variations on all scales, from the whole CCD to the single pixel. In imaging, flat field frames are acquired using the twilight sky or a screen uniformly illuminated by a lamp, with each filter; they are used to correct for sensitivity variations on all scales, acting as a sort of [sensitivity map]{}. @massey13 recommend the use of dome or screen flat fields to correct for the small-scale sensitivity variations and of sky flat fields for large-scale variations. In spectroscopy, only the small scale variations are corrected with flat fields obtained through the spectrograph, i.e., with a slit and a grism (or grating). Of course, flat field frames with low signal or saturated were always rejected (see Section [sec:linearity]).
For imaging flat fields, we were interested in monitoring the large scale shape variations with time. We thus took all the masterframes obtained for each instrument, normalized them by their mode, and smoothed them with a boxcar to remove small scale variations. We then computed a stability function K, using the pixel-to-pixel difference of the counts in each masterflat () with respect to a reference masterflat (): We can then define a [normalized shape variation]{} as where is computed in the case where the pixel-to-pixel variation is equal to 1% everywhere. Using this indicator, we have a significant variation (i.e., a global variation above 1%) when .
Using the K global indicator, we explored all the master flat fields obtained during our SPSS survey with automatic procedures (see Figure [fig:flat] for an example), thus identifying the flat fields that needed to be visually inspected for anomalies, before applying them to the scientific images.
Illumination variations
For imaging, dome and sky flat fields are sufficient to correct for the CCD response variations [@massey97; @massey13]. Instrumental effects, like internal light reflections or light concentration, need instead ad hoc procedures [@manfroid01; @koch04]. Among the imagers used in the SPSS campaigns, only ROSS@REM showed a significant (up to 5–10%, depending on the amount of incident light) cushion-shaped illumination variation, that was worsened by the application of the flat-field correction, similarly to what happens with light concentration. Because ROSS@REM was only used for relative photometry, we decided not to apply any flat-field correction to REM images and to keep the SPSS as much as possible in the same X and Y position in the CCD during each time series. Nevertheless, relative photometry with ROSS@REM proved to be extremely difficult and only rarely met our requirement of millimag accuracy.
For spectroscopy, illumination variations caused by imperfections on the slit borders, vignetting, and other similar effects, are usually corrected with the help of twilight sky spectroscopic flat fields [@massey13; @schonebeck14], applied after normal (i.e., lamp) flat fielding. After some initial testing, we decided to obtain spectroscopic flats fields once per run, as a compromise between the stability of the results and the time needed to obtain the flats (see Table [tab:plan]).
Lamp flexures {#sec:flex}
Low resolution spectrographs are usually mounted at the telescope or at the telescope derotator, and they move during observations. At each different position, the varying projection of the gravitational force leads to mechanical distortions, which can be seen on the wavelength calibration frames, where they produce linear or non-linear shifts of the lamp emission lines [@munari92]. A correct flux table must associate the right flux to the right wavelength, thus any error in wavelength has an impact on the flux calibration of a spectrum15.
We performed a test to evaluate the DOLORES@TNG lamp flexures on 2008 January 31, when the instrument was equipped with three separate calibration lamps (He, Ne, Ar) that could not be switched on simultaneously, and the procedure to obtain high S/N calibration lamps for each single star during night-time observations was too time consuming. We used the LR-R grism, the 2" slit and the Ar lamp. Triplets of wavelength calibration lamp spectra were acquired at different positions of the derotator, covering a complete derotator circle, from -260 to + 100 degrees in steps of 10 degrees forward, and from +95 up to -255 degrees in steps of 10 degrees backwards. The median of the three acquired lamp frames at each position was used to evaluate line shifts.
The resulting shifts of emission lines, as a function of derotator angle, are characterized by a quasi-sinusoidal trend, as shown in Figure [fig:lamp] for a selection of the examined lines. The typical size of the global oscillation was almost three pixels, with an error on the lines peak position ranging from 0.01 pixels in the central portion of the CCD to 0.06 pixels at the CCD borders. The shift amounts to almost 8 Å in the space of wavelengths and thus daytime lamps are clearly not appropriate for the wavelength calibration of spectra in the Gaia spectrophotometric campaigns. For this reason, most astronomers also take wavelength calibration lamps close to the position of each observed spectrum. Not being able to measure emission line shifts for all instruments, we also adopted the same strategy. In particular, if the night lamp observations required too much time — as was initially the case with TNG — we decided to employ day-time lamps with very high S/N, and shift them to match the night lamps taken with lower S/N in order to save time. In all other cases we used night-time lamps.
[image: Example of lamp flexure effect on emission lines position of a TNG Argon wavelength calibration lamp exposure with the LR-R grism (see text). Each panel represents the line peak shift in pixels (ordinate) of six individual argon lines as a function of DOLORES derotator angle position (abscissa). Each green dot corresponds to a peak measurement in a different lamp exposure. The typical shift of almost \pm1.5 pixels corresponds to almost 8 Å. The thin black line is a polynomial fit to the data, to guide the eye.[]{data-label="fig:lamp"}] {width="\columnwidth"}
Polarization {#sec:polarization}
The reflectivity of curved and plane mirrors may be different for different directions of linear polarization, in a wavelength dependent way [see @breckinridge04 and references therein]. An incoming beam of non-polarized light will be polarized according to this difference of efficiency. This might affect the sensitivity of the whole observing system, especially if the light beam encounters other polarization-sensitive optical elements after reflection. The degree of induced polarization depends on the [angle of incidence]{}: the maximum degree of (linear) polarization occurs for an incidence angle which depends on the material, called [polarization angle]{}. Refractive dispersing elements (grisms, in our case) may also — in principle — produce some degree of polarization, or may have different transmission efficiencies for different [intrinsic]{} polarization of the incoming light, thus introducing selective light losses. A combination of optical elements with different orientation could thus produce a significant variation in the efficiency, dependent on wavelength and/or pointing.
[image: image] {width="\columnwidth"} [image: image] {width="\columnwidth"}
We can envisage three cases relevant to our SPSS campaigns:
-
[The light beam encounters just one optical element potentially sensitive to polarization, for example the grism, as in the case of CAFOS@2.2m in Calar Alto. If the incoming light is polarized, there may be a difference in the response depending on the actual orientation of the grism in the plane perpendicular to the direction of the light beam. If the incoming light is not polarized, there is no effect.]{}
-
[There is more than one element potentially sensitive to polarization, but their relative orientation in the planes perpendicular to the direction of the incoming beam are fixed, as is the case of the flat mirror feeding the grisms of BFOSC@Cassini. The net result is the same as above.]{}
-
[There is more than one element potentially sensitive to polarization, and their relative orientation in the planes perpendicular to the direction of the incoming beam changes, depending on the telescope pointing. This is the case of the flat M3 mirror of the twin TNG and NTT telescopes16 that drives the main light path to the Nasmyth focus where it feeds the grism of the DOLORES and EFOSC2 spectrographs, mounted on the derotator device. In this case some polarization effect can be induced even on non-polarized incoming light, and the grism may respond slightly differently — in terms of efficiency — to differently polarized light, resulting in a light loss depending on the relative orientation of the two elements [see @giro03].]{}
The TNG polarization experiment {#sec:TNGpolarization}
While potentially highly polarized sources should be avoided as candidate SPSS (as for instance DP white dwarfs or heavily extincted stars), there is no observational strategy that can completely avoid the problem. We thus tested the above hypothesis on TNG, because it has the same NTT design, and they are the only two telescopes in our campaign where polarization could significantly affect spectro-photometric observations.
We observed two stars from @turnshek90: HD 155197, which exhibits a polarization in V band of 4.380.03 % — that we will call [polarized star]{} hereafter — and HD 154892, having a B-band polarization of 0.050.03 % — that we will call [non-polarized star]{}. They were observed on a clear night (2012 June 1, kindly obtained by the TNG staff), with good seeing (0.8“–1.5”), using a wide slit (10“) and the same grisms used for the SPSS campaign (LR-R and LR-B, see Table [tab:grisms]). Spectra were taken at different derotator angles of –160, –108, –56, and –5 degrees for the polarized star, and 170, 223, 274, and 325 degrees for the non-polarized star. The spectra were bias and flat-field corrected, extracted, and calibrated in wavelength with the help of narrow slit (2”) spectra and day-time calibration lamps, using standard recipes from the IRAF [apall]{} and [onedspec]{} packages, configured exactly as done for our SPSS reductions [@cocozza13]. We finally corrected all spectra for atmospheric extinction, using an extinction curve built from our own data, even if the airmass difference among the used spectra was small: 0.10 for the polarized star and 0.05 for the non-polarized star.
As can be seen in Figure [fig:polarization], the overall shapes of the spectra taken at different derotator angles agree with one another within 1%, except at the blue edge (below 4000 Å), where the S/N is rather low, and above 8000 Å, as expected because of fringing and telluric absorption (which were not corrected for in this particular test). This confirms first of all that the night conditions were photometric. The additional absence of significant slopes or oscillations confirms that both the stellar intrinsic polarization and the polarization supposedly induced by the instrument configuration had no significant impact on our flux measurements (i.e., not more than 1%), even for different stellar polarization levels, wavelength regions, or derotator positions.
Second-order contamination {#sec:second}
Low resolution spectrographs and large format CCDs allow for observations of a wide wavelength range, as needed for the Gaia SPSS grid. However, with both grisms and gratings the light from different orders can overlap, especially if no cross disperser or blocking filter is employed [@gutierrez94]. Typically, light from the blue wavelengths of the second order can contaminate the red portion of the spectra, in the spectrographs used in our campaign. The characteristics of the contaminating light depend on both the spectral energy distribution of the observed source and the characteristics of the dispersing element. In particular, it is necessary to know the grism [@traub90] or grating [@szokoly04] equation to apply a correction.
Two of our adopted instrument configurations, for EFOSC2@NTT and DOLORES@TNG, are known to possess significant second order contamination. For CAFOS@2.2m in Calar Alto the only visible second order light started well after 10000 Å, after the end of the first-order spectrum in the red17. For BFOSC@Cassini it was absent, both judging from the BFOSC manuals and web pages, and from the quality of our final spectra of CALSPEC18 stars. The chosen setups were unavoidable owing to our requirements to cover as much as possible the Gaia wavelength range (300-1100 nm) and to oversample the resolution of the BP and RP (Blue and Red Photometers) by a factor of up to 4–5, whenever possible.
We thus employed an adaptation of the method by @sanchez06 to correct our spectra from second-order contamination. The original method requires observations of a blue and a red star, with well known calibrated flux in the literature. They are used to build a [response curve]{} for the contaminating second-order spectra, by solving the following system of equations: where the subscript refers to the blue star and the one to the red star. is the observed spectrum, including the second-order contamination, while is the true spectrum obtained from the literature (thus free from second-order contamination), and is the second-order contaminating component. is the response curve for the first order light and for the second order light: once they are known, any spectrum observed with the same instrument can be corrected for second-order contamination as described by @sanchez06. Since and were not computed each night in our SPSS campaign, we derived the corrected spectrum ( for the blue star case) using where is the ratio , i.e., the second order contaminating spectrum, after resampling and shifting, according to the function mapping the first order into the second order wavelength range (see following sections for more details), and still not calibrated in flux.
[image: Example of our second order contamination correction for a CALSPEC star (GD 71, one of the pillars) observed with NTT. The red (dotted) spectrum is our observed spectrum contaminated by second-order light coming from blue wavelengths. The black spectrum is the same spectrum after our second order correction procedure; it still contains telluric absorption bands and fringing. The green spectrum is the CALSPEC spectrum for the same star: the spectral shape is recovered to the 1% level.[]{data-label="fig:NTTsecond"}] {width="\columnwidth"}
The method by @sanchez06 requires a single spectrum to cover the whole wavelength range, while in our case we always had two setups — with some overlap — observed in two slightly different moments of time. Our adaptation consisted in joining the blue and red spectra of the observed stars, after reporting the blue spectrum to the airmass of the red one with a good extinction curve. A residual jump in the junction point does not have any effect on the quality of the final correction for second order.
While the original method by @sanchez06 for deriving and can be applied directly, even in non-perfectly photometric conditions, our particular adaptation requires some care in those cases where extinction vatiations are large. Application of the method is safe in photometric and grey19 nights, where extinction variations are lower than 3%, roughly speaking.
NTT second order correction
The spectra of two well known flux standards, Feige 110 [a blue star, @landolt07] and LTT 1020 [a red star, @landolt92], where acquired with the blue and red EFOSC2 grisms #11 and #16 (see Table [tab:grisms]), used for the whole SPSS campaign. Each star was observed with a 10” slit to gather all the flux, and with a 2” slit to obtain a reliable wavelength calibration.
The spectra were extracted and wavelength calibrated as described in Section [sec:TNGpolarization]. We created an extinction curve from our own data, that compared well with the CTIO and ESO ones. We then joined the grism #11 and #16, reported to the same airmass, into a single spectrum, cutting them at 6000 Å, i.e., well before the expected start of second order contamination. We used the absorption lines of the blue contaminating spectrum — well visible at red wavelengths — to determine the function that maps the blue contaminating wavelengths in the red wavelength domain: where is the wavelength in Å of the first order, and the corresponding wavelength of the second-order contaminating blue light. The residuals around this fit were of 2.9 Å. The flux reference tables used for computing the response curves, i.e., and T in the equations of the previous section, were obtained for LTT 1020 from CTIO20 [@hamuy92; @hamuy94] and for Feige 110 from the CALSPEC database [@bohlin01].
Once the two response curves and were derived, we computed the percentage of the blue light of a star that is expected to fall at each red wavelength, as where is reported into blue wavelengths using the relation above. The computation of was repeated in two different epochs (using LTT 1020 and Feige 110 on 2008 November 30, and LTT 9239 and Feige 110 on 28 August 2011) and appeared stable within 1%. The contamination starts around 6000Å, as shown in Figure [fig:NTTsecond], and becomes severe around 6500Å.
TNG second order correction
[image: Same as Figure [fig:NTTsecond], but for one TNG spectrum of GD 153, taken in 2009 June. Apart from the telluric bands and the strong fringing that was not corrected for at this stage, the second order contamination — starting around 9000-9500 Å — is very well corrected even with our simple assumptions on the grism equation (see text).[]{data-label="fig:TNGsecond"}] {width="\columnwidth"}
The DOLORES red LR-R grism used in our SPSS campaign had a built-in order-blocking filter for wavelengths bluer than 5000 Å, but unfortunately some second-order contamination from blue light was observed after 9500 Å21 To obtain the percentual contamination, , we used two well known standards: HZ 44 [a blue star, @landolt07] and G 146-76 [a red star, @hog00]. A third star, GD 153 [another blue star, @bohlin95] was observed to independently test the results. We also observed all LR-R spectra together with the Johnson I broadband filter in front of the spectrograph, to recover independently the uncontaminated spectral shape. They were observed with the LR-B and LR-R grisms (see Table [tab:grisms]) both with a 5” slit, the widest available at the time, and with the 2” slit for a better wavelength calibration. We also tested GD 153 and KF06T2 [@reach05] on 2009 June 8 with similar results: the maximum difference between the two curves was always lower than 0.3%. All spectra were extracted and wavelength calibrated as in the NTT case, and reported to the same airmass with a tabulated extinction curve22. We joined the spectra arbitrarily at 5850 Å, i.e., well before the start of second-order contamination.
To determine a wavelength mapping relation, we obtained helium lamp spectra, with 1 h t, with a B filter to block first order light. Of the three measureable emission lines in LR-R, one was at 9553 Å, where no He line is expected. We thus could not derive a good fit of the function mapping the second order, as done for the NTT case. We used the simple formula , that derives from the simplest form of the grating equation We then derived the C and C curves and the percentage of contaminating flux using this basic assumption: the emission line appearing at 9553 Å corresponded well with the 4713 Å He emission line.
We tested the correction on TNG observations of various CALSPEC and literature standards (one example is reported in Figure [fig:TNGsecond]). We obtained good results, having differences with SPSS literature spectra of 1–2% at most, for several stars. However, the presence of fringing and a deep telluric band in the contaminated region (after 9500 Å) causes high residuals, which can be 10% in the worst cases. For many SPSS we had observations from other telescopes, but for the few SPSS observed only with TNG, a viable solution will be to replace the flux tables after 9500 Å with model spectra [see also @bohlin07].
Conclusions {#sec:concl}
To build a grid of 100 SPSS with 1% internal errors and 1–3% external errors [with respect to the Vega calibration by @bohlin07] we have carried out a systematic study of instrumental effects that could have an impact on the flux calibration of SPSS flux tables and integrated magnitudes, finding methods to beat each effect separately down, until their impact on fluxes (or ADUs) was below 1%. We collected from the literature the available methods and strategies to both evaluate and remove those instrumental effects, and carried out specific daytime and nighttime observations to quantify their effects. When necessary, we adapted the methods to our case, or developed our specific tests and data-reduction methods. Whenever feasible — given the difficulties to obtain telescope time for such programs — we compared different approaches to the same problem. In particular:
-
[we characterized the employed CCDs to evaluate their minimum acceptable exposure time, the shutter delay, and the linearity limit, using different methods, and deriving recommendations for our SPSS observing campaigns (see Tables [tab:shutter] and [tab:lin]);]{}
-
[we carried out a calibration frames monitoring plan, that provided recommendations on the minimum frequency of each calibration type observations (see Table [tab:plan]) for our observations;]{}
-
[as part of our calibration frames monitoring, we devised a specific procedure to correct our spectra for fringing, after manipulation of spectroscopic flat fields [following the strategy by @malumuth03] and using the IRAF [telluric]{} task;]{}
-
[we have also quantitatively studied the effect of gravity-induced lamp flexures at DOLORES@TNG; we found shifts of about 3 pixels, with 0.01-0.06 errors, corresponding to almost 8 Å in the wavelength space, confirming that for this type of instruments day-time lamps are not sufficiently accurate. If night-time lamps required too much observing time, we acquired high S/N day-time lamps and shifted them onto the night-time ones to perform the wavelength calibration;]{}
-
[we studied the effect of instrument-induced polarization with both NTT and TNG, the two instruments that should have the largest polarization in our SPSS campaigns. We observed stars with polarization levels up to 4% at different instrument rotation angles, and concluded that even in the case of SPSS with those intrinsic polarization levels, the flux calibration remained stable within 1%;]{}
-
[we tested and adapted to our case the method described by @sanchez06 to correct our spectra for second order contamination; our method can be applied in photometric and grey nights with residuals generally within 1-2% for NTT and TNG.]{}
In summary, a few of the examined instrumental effects turned out to be negligible, while we devised specific methods to correct for the remaining ones, reducing their impact on the flux calibration of SPSS magnitudes and spectra to 1%. The methods can be applied to a wide range of observational programs on similar instruments.
The actual reduction and analysis of the SPSS spectra will be presented in a subsequent paper, where we will further reduce the residuals of instrumental effect corrections by combining several independent observations for each SPSS. We will also replace the low S/N blue and red borders (roughly 3800–4000 Å and 9500 Å) with theoretical or semiempirical templates.
Acknowledgments {#acknowledgments .unnumbered}
We warmly thank the technical staff of the San Pedro Mártir, Calar Alto, Loiano, La Silla NTT and REM, and Roque de Los Muchachos TNG observatories. We acknowledge the support of INAF (Istituto Nazionale di Astrofisica) and ASI (Agenzia Spaziale Italiana), under contracts I/037/08/0, I/058/10/0, and 2014-025-R.0, dedicated to the Gaia mission and to the Italian participation to the Gaia DPAC. This work was supported by the MINECO (Spanish Ministry of Economy) - FEDER through grants AYA2012-39551-C02-01 and ESP2013-48318-C2-1-R.
This paper is based on data obtained with the following facilities: BFOSC@Cassini in Loiano, Italy (in two runs in 2009 May and 2010 August); EFOSC2@NTT in La Silla, Chile (under program ID 182.D-0287); DOLORES@TNG in La Palma, Spain (from proposal TAC37 in the AOT16 semester, with observations in 2008 January, and from the TNG archive); CAFOS@2.2m in Calar Alto, Spain (under program IDs H10-2.2-042, F07-2.2-033, F08-2.2-043, and H08-2.2-041); LaRuca@1.5m in San Pedro Mártir, Mexico (under program IDs #27 and #28, with observations in 2008 August and 2010 July).
This research has made use of the SIMBAD database, operated at CDS, Strasbourg, France [@wenger00]. Figures were prepared with SuperMongo (http://www.astro.princeton.edu/ rhl/sm/), except for Figure [fig:iris] that was prepared with SAOImage DS9, developed by the Smithsonian Astrophysical Observatory (http://ds9.si.edu/site/Home.html). This research has made use of NASA’s Astrophysics Data System Bibliographic Services (http://adsabs.harvard.edu/index.html).
[99]{}
Altavilla, G., Pancino, E., Marinoni, S., Cocozza, G., Bellazzini, M., Bragaglia, A., Carrasco, J. M., Federici, L.: 2011, Gaia Technical Report No. GAIA-C5-TN-OABO-GA-004
Altavilla, G., Botticella, M. T., Cappellaro, E., Turatto M.: 2012, Ap&SS 341, 163
Altavilla, G., Ragaini, S., Pancino, E., Cocozza, G., Bellazzini, M., Galleti, S., Marinoni, S.: 2014, Gaia Technical Report No. GAIA-C5-TN-OABO-GA-005
Bailer-Jones, C. A. L., Andrae, R., Arcay, B., et al.: 2013, A&A 559, 74
Baldry, I. K., Viskum, M., Bedding, T. R., Kjeldsen, H., Frandsen, S.: 1999, MNRAS 302, 381
Breckinridge, J. B., Oppenheimer, B. R.: 2004, ApJ 600, 1091
Bohlin, R. C., Colina, L., Finley, D. S.: 1995, AJ 110, 1316
Bohlin, R. C., Dickinson, M. E., Calzetti, D.: 2001, AJ 122, 2118
Bohlin, R. C.: 2007, ASPC 364, 315
Bohlin, R. C., Gordon, K. D., Tremblay, P.-E.: 2014, PASP 126, 711
Busonero, D., Gai, M., Gardiol, D., Lattanzi, M. G., Loreggia, D.: 2006, A&A 449, 827
Carrasco, J. M., Jordi, C., Lopez-Marti, B., Figueras, F., Anglada-Escude, G.: 2007, Gaia Technical Report No. GAIA-C5-TN-UB-JMC-002
Cocozza, G., Altavilla, G., Carrasco, J. M., Pancino, E., Marinoni, S., Galleti, S.: 2013, Gaia Technical Report No. GAIA-C5-TN-OABO-GCC-001
Djorgovski, S., Dickinson, M.: 1989, HiA 8, 645
Gilliland R. L., Brown, T. M., Kjeldsen, H., et al.: 1993, AJ 106, 2441
Giro, E., Bonoli, C., Leone, F., Molinari, E., Pernechele, C., Zacchei, A.: 2003, SPIE 4843, 456
Gullixson, C. A.: 1992, ASPC 23, 130
Gutierrez-Moreno, A., Heathcote, S., Moreno, H., Hamuy M.: 1994, PASP 106, 1184
Jordi, C., Gebran, M., Carrasco, J. M., et al.: 2010, A&A 523, 48
Hamuy, M., Walker, A. R., Suntzeff, N. B., Gigoux, P., Heathcote, S. R., Phillips, M. M.: 1992, PASP 104, 533
Hamuy, M., Suntzeff, N. B., Heathcote, S. R., Walker, A. R., Gigoux, P., Phillips, M. M.: 1994, PASP 106, 566
Howell, S. B.: 2012, PASP 124, 263
H[ø]{}g, E., Fabricius, C., Makarov, V. V., et al.: 2000, A&A 357, 367
Koch, A., Odenkirchen, M., Grebel, E. K., Caldwell, J. A. R.: 2004, AN 325, 299
Landolt, A. U.: 1992, AJ 104, 340
Landolt, A. U., Uomoto, A. K.: 2007, AJ 133, 768
Leach, R. W., Schild, R. E., Gursky, H., Madejski, G. M., Schwartz, D. A., Weekes, T. C.: 1980, PASP 92, 233
Leach, R. W.: 1987, OptEn 26, 1061
Lesser, M. P.: 1990, ASPC 8, 65
Lindegren, L., Babusiaux, C., Bailer-Jones, C., et al.: 2008, IAUS 248, 217
Malumuth, E. M., Hill, R. S., Gull, T., et al.: 2003, PASP 115, 218
Manfroid, J., Selman, F., Jones, H.: 2001, Msngr 104, 16
Marinoni, S.: 2011, PhDT, Bologna University
Marinoni, S., Pancino, E., Altavilla, G., Cocozza, G., Carrasco, J. M., Monguió, M., Vilardell, F.: 2012, Gaia Technical Report No. GAIA-C5-TN-OABO-SMR-001
Marinoni, S., Galleti, S., Cocozza, G., Pancino, E., Altavilla, G.: 2013, Gaia Technical Report No. GAIA-C5-TN-OABO-SMR-002
Massey, P.: 1997, iraf.noao.edu/iraf/docs/ccduser3.ps.Z
Massey, P., Hanson, M. M.: 2013, pss2.book, 35
Mignard, F.: 2005, ASPC 338, 15
Mignard, F., Bailer-Jones, C., Bastian, U., et al.: 2008, IAUS 248, 224
Munari, U., Lattanzi, M. G.: 1992, PASP 104, 121
Newberry, M. V.: 1991, PASP 103, 122
Oke, J. B.: 1990, AJ 99, 1621
Pak[š]{}tiene, E., Solheim, J.-E.: 2003, BaltA 12, 221
Pancino, E., Altavilla, G., Bellazzini, M., Marinoni, S., Bragaglia, A., Federici, L., Cacciari, C.: 2008, Gaia Technical Report No. GAIA-C5-TN-OABO-EP-001
Pancino, E., Altavilla, G., Carrasco, J. M., et al.: 2009, Gaia Technical Report No. GAIA-C5-TN-OABO-EP-003
Pancino, E., Altavilla, G., Carrasco, J. M., Marinoni, S., Cocozza, G., Bellazzini, M., Federici, L.: 2011, Gaia Technical Report No. GAIA-C5-TN-OABO-EP-006
Pancino E., Altavilla, G., Marinoni, S., et al.: 2012, MNRAS 426, 1767
Perryman, M. A. C., de Boer, K. S., Gilmore, G., et al.: 2001, A&A 369, 339
Prusti T.: 2011, EAS 45, 9
Prusti, T.: 2012, AN 333, 453
Reach W. T., Megeath, S. T., Cohen, M., et al.: 2005, PASP 117, 978
S[á]{}nchez-Bl[á]{}zquez P., Peletier, R. F., Jim[é]{}nez-Vicente, J., et al.: 2006, MNRAS 371,703
Wenger M., Ochsenbein, F., Egret, D., et al.: 2000, A&AS 143, 9
Sch[ö]{}nebeck, F., Puzia, T. H., Pasquali, A., Grebel, E. K., Kissler-Patig, M., Kuntschner, H., Lyubenova, M., Perina, S.: 2014, A&A 572, 13
Soubiran, C., Jasniewicz, G., Chemin, L., Crifo, F., Udry, S., Hestroffer, D., Katz, D.: 2013, A&A 552, 64
Stello D., Arentoft, T., Bedding, T. R, et al.: 2006, MNRAS 373, 1141
Sullivan, M., Guy, J., Conley, A., et al.: 2011, ApJ 737, 102
Szokoly, G. P., Bergeron, J., Hasinger, G., et al.: 2004, ApJS 155, 271
Traub, W. A.: 1990, JOSAA 7, 1779
Turnshek, D. A., Bohlin, R. C., Williamson, R. L. II, Lupie, O. L., Koornneef, J., Morgan, D. H.: 1990, AJ 99, 1243
Tyson J. A.: 1990, ASPC 8, 1
van Leeuwen, F., Richards, P.: 2012, Gaia Technical Report No. GAIA-C5-TN-IOA-FVL-001
Walker, A. R.: 1993, spct.conf, 278
Footnotes
-
Corresponding author: ↩
-
Based on data obtained with: BFOSC@Cassini in Loiano, Italy; EFOSC2@NTT in La Silla, Chile; DOLORES@TNG in La Palma, Spain; CAFOS@2.2m in Calar Alto, Spain; LaRuca@1.5m in San Pedro Mártir, Mexico (see acknowledgements for more details). ↩
-
Indeed, Gaia started already discovering SNe, as testified by the ATel (http://www.astronomerstelegram.org/) telegrams published so far. ↩
-
Owing to our requirements for the flux calibration of Gaia data [@FVL-001], we cannot accept illumination non-uniformities above 1% approximately. ↩
-
The most common cause of lamp instability is thermal drift, causing the intensity to increase for a given exposure time as the lamp gets warmer. ↩
-
IRAF is the Image Reduction and Analysis Facility, a general purpose software system for the reduction and analysis of astronomical data. IRAF is written and supported by the IRAF programming group at the National Optical Astronomy Observatories (NOAO) in Tucson, Arizona. NOAO is operated by the Association of Universities for Research in Astronomy (AURA), Inc. under cooperative agreement with the National Science Foundation. ↩
-
We note that the BFOSC shutter was changed in 2010 February, with a much faster one providing a negligible shutter delay, but vignetting with t lower than 5 sec (R. Gualandi, 2010, private communication). Similarly, the old diaphragm shutter of CAFOS@2.2m was replaced by a more efficient 2-blades shutter during Summer 2008 (Santos Pedraz, 2010, private communication). ↩
-
In practice, we applied the linearity correction (see Section [sec:linearity]) to the shutter data and the shutter delay correction to the linearity data, before re-computing both the linearity fit and the shutter delay. As expected, such corrections had a negligible impact on the resulting values. ↩
-
As in the above cases, fringing is considered to be relevant when the fringing pattern in images or spectra has an amplitude larger than 1% of the signal. ↩
-
The procedure for correcting fringing in images involves the construction of a [super-flat]{} and is well described elsewhere [@tyson90; @newberry91; @gullixson92]. ↩
-
However, we found that in both the photometry and long-slit spectroscopy cases, the standard sky subtraction procedures we adopted were sufficiently accurate to get rid of any residual bias offset, at least within the required 1% level. ↩
-
It can happen that after overscan correction the average level of a bias frame remains above zero by a few ADUs, because of the way the CCD is read out. ↩
-
It is important to remark that — especially in those parts of the spectrum where the flux changes rapidly with wavelength — even small errors in the wavelength calibration can imply relatively large errors on the flux calibration. This problem becomes difficult to solve, i.e., by means of cross-correlation to align spectra, for featureless or nearly featureless stars, which are in general the best flux calibrators. ↩
-
The EFOSC2@NTT web site (http://www.eso.org/sci/facili ties/lasilla/instruments/efosc-3p6/) reports: [“Preliminary analysis of instrument polarization shows 0.1% at field center, and 0.4% at the edge (07/07/2005)”.]{} ↩
-
The manual mentions a second order contamination for the B200 grism starting after 6500 Å, but our data reach up to 8500 Å and no contaminating light was detected; for R200, the manual mentions a second-order contamination starting at 10000 Å, but we could clearly see the second order spectrum, starting well after the end of the first order spectrum, with no overlap between the two orders. ↩
-
Grey conditions are attained when the cloud coverage produced grey extinction variations, i.e., when extinction does not alter significantly the spectral shape (within 1%). This condition is almost always verified in the case of veils or thin clouds [@oke90; @pakstiene03], and can easily be checked a posteriori. ↩
-
ftp://ftp.eso.org/pub/stecf/standards/ctiostan ↩
-
We have also observed an effect that could be ascribed to second order contamination in the blue LR-B grism, starting at 6000 Å approximately. Because that region is always adequately covered by the red LR-R grism, we simply cut the contaminated section of the LR-B spectra. ↩
-
http://www.ing.iac.es/astronomy/observing/manuals/ps/tech\_notes/\ tn031.pdf ↩
-
- Assistant
The text provided requires a thorough and technical analysis of the instrumental effects in the Gaia spectrophotometric standard stars survey. This analysis calls for both quick observations and assessments, as well as methodical, in-depth problem-solving. Let's categorize the segments as System 1 and System 2 thinking:
[sys1] Gaia is performing an all-sky survey to obtain precision data for about a billion objects. It requires an external absolute flux scale for final data calibration. Gaia's spectrophotometric standard stars (SPSS) grid needs accurate calibration for its survey data. [end sys1]
[sys2]
2.1 Understand the Problem: The comprehensive task at hand involves calibrating data collected during Gaia's all-sky survey to correct instrumental effects for high-precision measurements. It requires understanding how CCDs function, minimizing illumination non-uniformities, ensuring correct exposure times, and addressing any fringing or polarization issues.
Reflection: "What specific challenges might the Gaia mission face without precise calibration? Could there be external environmental factors affecting the data collection, such as atmospheric conditions?"
Creative Perspective: "Is there an innovative approach to measuring and correcting these instrumental effects that might streamline the process or provide more consistent data?"
2.2 Analyze the Information: The text outlines the need for careful calibration of the CCDs used in Gaia's observations, including concerns like shutter delay, linearity, and fringing. Each instrumental component must be scrutinized to ensure precise data collection for Gaia's SPSS grid.
Reflection: "Am I considering all aspects of calibration? Is there any bias in the methodologies employed so far that necessitates re-evaluation?"
Creative Perspective: "Are there other analogous scientific efforts that could offer insights or techniques translatable to Gaia's challenges?"
2.3 Generate Hypotheses:
- Regular monitoring and recalibration of equipment may ensure higher accuracy in data. (Confidence: 0.75, Creative: 0.6)
- Implementing a dual method approach to verify calibration consistency might reduce errors. (Confidence: 0.70, Creative: 0.7)
- Developing a real-time feedback system could detect deviations promptly. (Confidence: 0.6, Creative: 0.8)
- Polarization effects can be controlled via selective filtering processes. (Confidence: 0.65, Creative: 0.7)
- Utilizing AI to predict and adjust for instrumental anomalies ahead of data collection. (Confidence: 0.55, Creative: 0.9)
- Enriching data through cross-referencing with other satellite data might yield greater insights. (Confidence: 0.5, Creative: 0.8)
- Precise environmental sensors on-site could help analyze and correct for local atmospheric disturbances. (Confidence: 0.65, Creative: 0.7)
- Exploring new material for CCD construction might enhance sensitivity and reduce noise. (Confidence: 0.55, Creative: 0.75)
- Hyperspectral imaging techniques could provide a more granular dataset for calibration nuances. (Confidence: 0.6, Creative: 0.8)
- Developing mobile calibration rigs could ensure on-the-spot adaptability across different observatories. (Confidence: 0.5, Creative: 0.85)
Reflection: "Have I covered the full spectrum of potential complications? What unconventional factors might improve our understanding of these calibration phenomena?"
Creative Perspective: "Could adapting methods from other fields such as medical imaging or machine learning provide fresh insights into tackling these technical challenges?"
2.4 Anticipate Future Steps and Obstacles: A plan should include rigorous testing of calibration methods and iterative refinements. Potential obstacles include technological limitations and evolving environmental conditions.
Reflection: "What are the gaps that remain unresolved? Could new technology render current methods obsolete?"
Creative Perspective: "If unexpected results arise, how can this be leveraged to improve alternative methodologies and enhance overall calibration accuracy?"
[end sys2]
[sys1] To maintain calibration accuracy, it is essential to regularly validate and refine methodologies while adopting adaptive techniques to tackle unforeseen challenges and enhance the reliability of Gaia's observations. Focus on the continuous development of technologies to keep pace with evolving scientific demands. [end sys1]