NMAEA Conference - LACES

New Mexico

NMAEA Conference

November 2014

1. How can I use the searches and reports in LACES to improve program efficacy?

There are many ways to use the software to improve program efficacy. You can save reports from quarters/prior fiscal years and re-run them to see if you’re doing as well as last year, or to compare to see if you’re on target to meet your benchmarks. All NRS tables can be run for prior Program Years if you start from All Students and then go to Print>Print Reports and click on the Table you want to run, then enter the prior Program Year selection.

You can also use recommended data review searches and/or views to be sure data is being entered appropriately.

It is strongly encouraged that agencies review their data regularly. Some recommendations for data review and the instructions for obtaining the information follow.

·         Number of Current FY Students

·         Number of NRS participants

·         Number of NRS participants on Tables 1-4 & totals for Table 6

·         Number of non-NRS participants and reasons why

·         Number of NRS participants post-assessed/number of those who made level gains/number of students with too few hours between assessments (check Comments report)

·         Number of NRS participants with enough hours for post-testing but did not receive a post-test (including people who left with enough hours, excluding those with HAS Entry Level and excluding those who already made a level gain in FY)

·         Current FY students missing new fields

·         Current FY students still enrolled with no hours in 90+ days

·         Staff missing new fields

Number of Current FY Students: From the Student grid, go to Find>Current FY. Look to see if the number is accurate. Look for students who have a status of Prospective with an Intake Date from more than 90 days ago. (Consider creating a custom view to add Intake Date.)

Custom views are extremely useful due to the wealth of information that they can provide at a glance.  Because custom views allow you to change the information displayed for whatever population you are working in, you can change a view and then use the Print functions to create an ad hoc report that can be printed via Print HTML, or saved and sent as an attachment via Print PDF or Print Excel.

To create a Custom View with Date of Intake, go to the View menu in the student grid. Select “Create New Views.”  You will get the “Add New Views” pop-up window.  From this window, give your view a name (use something that will help you remember what fields you modified for that view) and then select the “Fields” button. 

You will now see the Field Selection window.  In the left-hand column are all of the available fields that can be added to your view.  In the right-hand column are the fields currently being displayed.

You can add fields by clicking once on the field in the left-hand column to select it, then clicking the “Add” button.”  If you are adding multiple fields, you can hold down the CTRL button on the keyboard while making your selections to highlight more than one, but you will need to click the Add button once for each selection to add them. You can remove fields by clicking once on the field in the right-hand column and clicking the “Remove” button.  You can also change the order in which fields display by clicking the “Move Up” and “Move Down” buttons.  For this example, add the Intake Date field.  Once you have the fields you want for this view, in the order you want them displayed in, click “OK.”

You will be returned to the “Add New Views,” where you will be able to verify that the Fields now displays the options you added.  Additionally, from here you could select the “Sort By” button to pre-determine the order in which columns sort.  The default order is alphabetically by last name, but using the “Sort” function allows you to pre-order multiple columns at a time based on the criteria you choose.  Bear in mind that you can also always sort columns into ascending or descending numeric or alphabetical order by double clicking the column heading for ascending order, then clicking once more for descending order.

Custom Views are always saved once applied.  You can return to the default view by going to View>Key Info, or by logging out and back into the database.  A saved view can be re-applied at any time by going to View>My Views and locating the view by name.  Views are specific to user name, so if one person in your agency saves and uses a view, other users would still see whatever view they are using themselves. If you export or print data with a Custom View applied, the data in your custom view is what will export. Custom views are username specific, so other persons in your agency will not see them in their My View list.

If you have Prospective students with Intake Dates from 90+ days ago, consider editing their Overall Status from Prospective to Never Attended.

Number of NRS participants: From the Student grid, go to Find>Current FY, then Find>NRS>Fundables in the Selection. These are the students who are 16 years of age or older, with 12+ hours of instruction and a valid assessment generating a valid Educational Functioning Level (EFL) in the current FY.

Number of NRS participants on Tables 1-4 & totals for Table 6: From the above selection (or from Current FY student selection) go to Print>Print Reports. Bring up FY Data Table 1 (not Current Data Table, as this runs off raw data and is less accurate). Verify that the number in the Total column matches the number of NRS participants returned in your above search. Repeat with Tables 2-4. Repeat with Table 6, but rather than looking in the Total column on Table 6, verify that the 3 lines of Employment Status (Employed, Unemployed, Not in Labor Force) add up to the total of NRS participants, and that the columns and rows for Highest Education Level Completed at Entry and Location also add up to the total of NRS participants.

If any of the totals do not add up, return to the list of Current FY NRS Participants and go to Find>Fiscal Year Based Diagnostic Search to determine why.

The Fiscal Year Based Diagnostic Search is an extremely useful search for identifying issues in student data that can potentially prevent them from accurately populating your NRS tables. 

This search is run using summarized, fiscal year data, which means you want to run this search after LACES has run FY summaries, which is automatically done on a monthly basis, or after you have manually run FY summaries for your agency.

To run manual FY summaries, select your Current FY students by going to Find>Current FY. This will return those students who have been active at some in this FY. LACES can only run FY summaries on selections of 200 students or fewer, so if you have more than 200 students in your current FY selection, you’ll need to schedule the summaries to be run, or break the selection down to subsets of 200 and run them in groups.

To schedule the summaries, turn off paging and then click the top checkbox on the left hand corner of the student grid. This will select all current FY students. Go to Selection and Subset this group of students. Then, go to New and click on Create FY Summaries. You can see that the “Create FY Summaries Now” button is inactive, because the selection is larger than 200 students. Click the schedule button to schedule the summaries to be run by LiteracyPro. Generally, it takes 1-2 business days to run summaries. You will be notified via email when the summaries are run, provided your account has a valid email address associated with it. You can go to the User>Account tab and edit or add an email address if necessary.

If you want to break your selection down to subsets of 200 students so that you can manually run the summaries yourself, go to the User tab and click the My Settings folder. Here, you can change the number of records that display to 200. Now, when you subset one page of students, it will be the 200 displayed, allowing you to manually run summaries on each page of students.

Once summaries have been run, whether by LiteracyPro automatically, by being scheduled, or manually by you, you are ready to run a FY Based Diagnostic Search.

Generally speaking, it makes the most sense to run the Diagnostic Search on students who are NRS Participants. This search is intended to locate issues in student records that would populate the NRS tables if not for the errors.  So, we’ll start by going to Find>Current FY, then Find>NRS>Fundables in the Selection. These are the people who should be populating your NRS tables.

Now, go to Find>FY Based Diagnostic Search. Select the program year, and then select the option to search in the current selection. Apply this search condition. Go down the list on the right hand column, clicking Get Results for each option.

The first category is Search for Students Using Fiscal Year Data. The first row indicates students with FY data in the above reporting system. This should match the number of students in your selection, indicating that all students have a FY summary created. If it does not match, look at the results for Students Without FY data in the above reporting system. If there is a number here, click on that number and run a FY summary on the selection. Students without FY data will not populate your FY NRS tables. Student with FY data created before today in the above Reporting System means that the students have a FY summary, but it was created prior to today, meaning some data may not be as up to date as you’d like it to be. It’s best to run this search with data that has been created very recently.

The next category is the Search for Students missing Demographic Data in FY Records, and contains indicators of missing ethnicity/race information or gender information. It is very unlikely that you’ll see students reported in this section, since LACES required you to enter that data to save the record, but some agencies migrated data from other systems, so it is possible. If you see numbers in this category, click on the number. You’ll be taken to the list of students missing that data. You can open the record, go to the Demographics folder, edit the record to add the data, and save. Remember that you’ll need to run a new FY summary after any corrective changes made so that they will be picked up for the search and reporting purposes.

Next is the search for Students with Missing or Invalid Education Data. This is a very important search, as new fields were added effective July 1, 2012, that require education data for federal reporting. Again, these fields are required for new, incoming students, but many students who were prior FY students may have returned and because these fields weren’t required when they did their initial intake, you make have students missing these fields. Click on the number for students missing Highest Education Level Completed on Entry. You’ll be taken to the list of students missing that data. Open the first record and go to the Education folder. Click Edit in the bottom right hand corner, and edit the Highest Education Level Completed on Entry and Highest Education Level Completed on Entry Location fields, then click the Save & Return button. Run a new FY summary to be sure the updates have been captured.

The Search for Students with Incorrect or Missing Employment Data is another search that usually has little to no results, because Employment Status is required on intake. However, again, migrated data may cause issues, as can students returning from prior FYs who had invalid statuses, or incorrect dates that do not correspond to the intake date or are invalid dates. Click on the number for students found with this error, and go to the Work History folder to correct any issues. Save, and run a new FY summary.

The Search for Students with Incorrect Assessment or Level Data is one of the most common areas to see students reported as having issues. Students with Subject Area in Student Overall Status History not matching current subject area is a common technical support issue. These records are usually caused by assessments that were pushed forward from one year to the next, but the database did not populate a current FY record in the Overall Status History folder. Other issues can be caused if subject areas are changed after the initial assessment. Because the Overall Status History folder cannot be modified by local users, you will need to contact technical support to have us update these records if you have results in this category, as well as with the Students with Entry Level Student Overall Status History not matching current entry level, which tends to go along with the Subject Area search.

The Students with incorrect GED levels in FY records search picks up students who have incorrect data in their FY record. In order to show as having Completed High Adult Secondary they must have an entry level of High Adult Secondary.  This usually occurs if the Overall Status History record has no subject area from a non-GED pre-test.  If you have numbers here but do NOT have numbers in the NRS Not Reportable row, this may not need to be corrected as long as your NRS tables look accurate. To correct this, contact technical support.

Students with more than one assessment pushed forward in the Reporting Period: LACES allows you to move assessments from one FY to the next by using the “Move Forward” button in the Assessment record. This should be done within the guidelines of the State agency you report to, as well as the test manufacturer’s guidelines. However, you should never move more than one assessment forward in the same subject area. If more than one assessment has been pushed forward in the same subject area, those students will show up in this search. To correct the records: Delete extraneous pushed forward assessment records for the new FY. Only one assessment per subject area may be pushed forward (and there is no reason to push forward more than one subject area since the student can only be tracked in one subject area per year for federal reporting purposes.) Run a new FY Summary Record after making corrections to students returned in this search.

The Students with Current FY Instruction Hours totals not Matching the Recalculated Total search will bring up any students whose current FY hours totals do not equal the sum of their hours for the Fiscal year. Most often, this is because hours have been added since the most recent FY summary was created. This can be avoided by always waiting to run the FY Based Diagnostic Search until immediately after summaries have been run. To correct the records: Click on the number of students returned in the Get Results area. Run new FY summaries on these students and then run this search again. If the search is still returning students, contact Tech Support to run an hours recalculation on your data. Run a new summary for any students in this category BEFORE attempting to fix other NRS non-reportable students, as this often clears up non-reportables.

The Search for Students: Total Not Reportable for NRS search should be run after all the other data has been cleaned up. If you run this search only on NRS Participants and have no issues in any other areas, you should have no non-reportable students. If you run the search on non-participants, you may find other problems with records that cause them to not be counted that are not included in the other diagnostic searches. To correct those records, you will need to review the records for common issues including a lack of hours or a valid assessment in the current FY. If you cannot determine the problem then send the list to Technical Support for analysis.

Number of non-NRS participants and reasons why: Just as it is important to know who is an NRS participant, it is also important to know who is not, and most importantly, why they are not. Starting with your Current FY students, go to Fund>NRS>Fundables and Remove from Selection. You will be left with students not counting on your NRS tables.

Occasionally, data entry errors in the required assessment or demographic information will keep a student from being picked up as a participant as well. To check why these students are not counting, use the Custom View called “NRS” to look at their information this fiscal year.

First, go to View > NRS.

• Sort the CurFYInstructHours column by double-clicking the column heading. This will sort by ascending order. Click once more on the column heading to sort in descending order.
• Students with <12 hours you should not be concerned with, as they should not count as participants (although you may want to look for data trends to spot possible retention issues.)
• If a student has 12+ hours, check their AssessedStatus. If it says “Never Assessed this Fiscal Year” or “No Value Entered” the student has not been given as assessment, or the assessment was not a valid assessment as defined by NRS. You can check this by going into the student record, then going to the Assessments folder. Look for any errors that kept an assessment from counting.  Additionally, check to see if the student has an assessment from last fiscal year that could be moved forward to count for this FY. You can retroactively push forward an assessment if you missed one from last FY. You may want to do an Assessment Search>Assessment Status Search and search for students never assessed this year but who had assessments last year that could be pushed forward. Bear in mind that the Adult HS assessment is NOT a valid assessment according to the NRS; it has only been used for cohort generation and should never be the student’s only FY assessment.
• If the student has a valid assessment and 12 or more instructional hours, check the Demographics information to see if there is missing or incorrect data in the required fields, or if the student was not 16+ years of age at entry. Additionally, make sure students do not have the Primary Program Work-based Project Learner unless they really are a Work-Based Project Learner, as that prevents them from being counted as an NRS participant. Also check their Funding Stream to see if they are not supposed to be NRS students.

Number of NRS participants post-assessed/number of those who made level gains

To see the number of students post-assessed, go to the Navigation Pane on the far left. Click on the + sign to the left of the Assessment Status folder and then click the folder called Assessed 2+ times this FY.

Alternatively, you could use the Quick Field Search at the top of the screen and choose Assessment Status: Contains: 2, then use the magnifying glass on the right to search in your current selection of current FY students.

Once you have the list of students who have been assessed 2 or more times, you can then see how many have made level gains. You can go to Print>Print Reports>FY Data Table 4 from your current selection to view number of students in this selection who have gained, or you can use the Find>NRS Table Search>NRS Table 4, or you can simply use the Quick Field Search again and change the fields to Completed Level: Begins with (or Contains): Yes and use the magnifying glass on the right to search in your current selection.

In addition to checking who has been post-tested and who has made gains, you may want to verify that the students were post-assessed within the correct number of hours based on state policy.

Hours between assessments has become a big area of concern with OCTAE. Your state has an assessment policy regarding how many hours a student must have at a minimum in order to be eligible for post-assessment. These hours are required whether it is their first or second (or more) post-test within a fiscal year.  Typically, the hours between assessment minimums are:

At least 30 for ASE students (Low Adult Secondary)

At least 40 for ABE students

At least 50-60 for ESL students or ABE students with certain assessments

Most states do have exception policies for post-testing sooner, but if you do, you should use the Comments folder in the student record to document why. Familiarize yourself with your state’s assessment policy.

You can use the Assessment report called Hours Between Assessment to verify that your agency is following policy by going to the Student Grid, then using one of the options above to narrow you’re your selection to only post-assessed students. Then go to Print>Print Reports>Hours Between Assessment, click then click Print PDF or Print Excel.  You might also narrow down by program (ABE/ESL) or by choosing Entry Level>Contains>ABE (or ESL) (or Low Adult) since that will enable you to run the report with the specific hours options for those programs.

From the report, choose the appropriate number of hours, bearing in mind that you are searching for students who did not comply with the state policy, so you would want to search on students with fewer than the required number of hours.

The returned report will give you students who were post-tested outside of the state’s assessment policy for minimum number of hours.

Be sure to also run a Comments report to see that students on your Hours Between Assessments report are documented in the Comments folder with an explanation.

Number of NRS participants with enough hours for post-testing but did not receive a post-test (including people who left with enough hours, excluding those with HAS Entry Level and excluding those who already made a level gain in FY): There are many ways to locate students who are eligible for post-testing. You can use the Dashboard Performance Indicator called Students Needing Post-Testing, or use the Alert indication by the same name. You can use the Find menu in the student grid and go to Assessment Search>Assessment Status search and search based on hours, or the Quick Select Search and do the same. Use a range beginning with the minimum number of hours required. 

Assessment Status Search allows you to search for students who have not been assessed or whose last assessed date fell within a provided date range, as well as students who have more than x hours or y days since their last assessment. This search can be very useful for locating students eligible to be post-assessed, or those students who have assessments that are eligible to be pushed forward from one fiscal year to the next.

To generate this search:

1.       Go to the Student tab in the navigation pane of LACES.

2.       Choose Find from the menu bar and select Assessment Search>Assessment Status

Search

3.       Select the criteria on which you wish to search. You can choose to search for all students who have never been assessed, or whose last assessed date was within a date range that you choose. You can also choose to search for students who have more than x hours since their last assessed date, based on your choice of number of hours, or you can choose to search for students who have more than y days since their last assessed date, based on your choice of number of days.

4.       Choose one of the radio button selections for searching in all records or current selection.

5.        Once you have defined all the required parameters of your search, click the GO button to bring up the search results in the List View.

Once you have your list of students with enough hours to be post-assessed, you can choose to narrow it down further, if you choose to. For example, students with an ENTRY level of High Adult Secondary cannot be post-assessed using the same instrument, since they’ve come in at the highest ABE level, so you may want to use the Quick Field Search and choose: Entry Level: Does Not Contain: High Adult.

Additionally, you may choose to remove students whose status indicates they have already exited your program, or those who have already made a level gain. (Quick Field Search= Completed Level: Contains: No) You can, of course, choose to leave those students in your selection, as well.

Once you have this list, it can be helpful to use it to provide an Assessment Report to the people who will be post-assessing the students, letting them know that they are eligible for post-assessment, and what their last Assessment data was.

Current FY students missing new fields: Students with missing or invalid fields for the Highest Education Completed at Entry/Location fields added in 2012 will cause discrepancies from Tables 1-4 to Table 6. Use the FY Based Diagnostic Search described above in the section for NRS Participants on Tables 1-4 and 6 to locate and correct any missing data for these fields.

Current FY students still enrolled with no hours in 90+ days: Students should be properly exited in LACES at 90+ days without service, or at the time of their known exit. Use the Dashboard Indicator for Students Enrolled but Not Attending in >12 weeks or the Alert Indicator for Students with No Instructional Hours for X Days (with x=91) to locate students who need to be exited.

Students should be exited from the program based on known departure or 90 days without service; however, please be aware that LACES does not use the class end date entered as their end date to populate the Left Date field. Left Date is their last known date of attendance, based on Last Hours Date. The Left Date populates the Key Information section of the student’s record, and should be used for doing outcome follow-up for employment goals. Cohort data will give you the specific quarters of follow-up based on the left date, but goals data required a Met Date, which should be based on the LACES-generated Left Date.

You can create a custom view that shows the Left Date assigned by LACES. From the student tab, go to View>Create Custom View. Name your view, then click the Fields button. From the left hand column, locate Left Date and click that field, then click the Add button. When it has been added to the right hand column, click OK, then Save and Apply to Grid. Your view will change to display the Left Date column. Remember that the Left Date is based on their Last Hours Date.

You may notice that sometimes a Left Date will display for a student who is still Enrolled. This indicates that the student has gone more than 90 days without attendance hours being entered into their record, so even though you still have them as Enrolled, the database considers them left for reporting purposes. If you add hours for those students, the Left Date will be removed.

Staff missing new fields:

New staff data became required effective 7-1-12. They are:

·         The field called Years of Teaching Experience in Adult Education

·         Indicators of teaching credentials in the staff credentials folder

This data must be populated for all staff member who were active in your fiscal year, even if they were not teachers.

For Certification, OVAE is interested in tracking the following certifications:

·         Adult Ed

·         Special Ed

·         K-12

·         TESOL

If a staff member has any of those certifications, you should manually go to the Credentials folder and click the Add New button to add the certification Type and Date Earned. Both the Type and Date Earned are required for the information to correctly populate Table 7, which is a mandatory table. If the staff member does not have any of the 4 tracked certifications, you should enter No in the Certification field.  A staff member can have more than one credential type and each should be entered into the Credentials folder. If the staff member has additional certifications or credentials outside of the 4 tracked on Table 7, you should enter those based on your state policy.  The rows and columns for Teacher Certifications on Table 7 should at least equal the sum of the Local Teachers (FT and PT) row on Table 7 (excluding volunteers.)

For Years of Teaching Experience in Adult Education, this only refers to teaching experience in adult education. For staff members who are not teachers, this should be 0. Otherwise, enter the number of year’s experience teaching specifically adult education. This must be manually updated every year; it will not update automatically. The Years of Teaching Experience in Adult Education rows should add up to exactly the sum of the Local Teachers (FT and PT) row on Table 7 (excluding volunteers.)

The best way to verify this data is to create a custom view that adds the Years of Teaching Experience in Adult Ed field.

For this view, you should see no blank fields in the new staff information fields. If there are blanks for any staff member who was active in the FY, edit the Key Info folder in the Staff record to add those fields.

Updated data will not display on Table 7 until a new FY Summary has been run for the updated staff records. To run a new FY Summary, select the staff members by checking the box in front of their names on the Staff grid. Go to Selection>Subset. Go to New>Create FY Summary and click the Create FY Data Now button to create the summaries right then, or the Schedule button to have LiteracyPro run the summaries within 2-3 business days.

When viewing Table 7, the number of full and part time local teachers (excluding volunteers) should add up to the total sum of the Years of Teaching Experience in Adult Education rows and columns. The Certification rows and columns should add up to at least the sum of the full and part time local teachers (excluding volunteers), although since a staff member can have more than one certification, it may exceed the total. If your numbers are not reflecting as they should, please contact our technical support department.

Finally, your staff list should accurately reflect the active staff members in your agency. If people have left, mark them left. If people have been hired, add them. If you mark them left now and they actually left last year (or more years back), you must go into the Staff History folder and update both the End Date of the Active status and the Start Date of the Left status so that they do not show activity in the current fiscal year date range. Remember that the Years of Teaching Experience in Adult Education field must be manually updated at the beginning of each new fiscal year for teachers whose experience has increased.

2. How do we handle students who are both ABE and ESL? Does the Primary Program get updated, or just the Program folder, or both? What about the ESL checkbox?

The most important thing to understand about Programs is that, for the purposes of federal reporting, program is reported based on the student’s Entry Educational Functioning Level, not the Primary Program, Program folder, or ESL checkbox. Also, a student can be in one type of Primary Program for their student record, but be enrolled in classes with a different Primary Program and that’s okay, because what determines their population on Table 3 (NRS Participants by Program report) is their Entry EFL.  As an example, if you have a student whose Primary Program is ESL, but they came in too high (Completed Advanced ESL as an Entry EFL), so you gave them an ABE assessment and are using that ABE level as their Entry EFL, then they are going to populate Table in the Adult Basic Education row.  For reporting purposes, a student cannot be reported in multiple programs, but realistically, it happens that a student might be being tracked for one subject area in ABE/ASE and another for ESL, so LACES reports based on the evidence of the student’s Entry Educational Functioning Level.

For that reason, Primary Program is really only important for 2 reasons:

a)      To allow you (and/or the state) to easily location students within a specific program. You might want to know everyone who started as ESL and transitioned to ABE, or just want to know everyone in a specific program such as EL Civics. Populating a Primary Program allows you to use the Quick Field Search, Quick Select search, or Navigation Pane folder of Primary Program to easily locate or sort those students.

b)      Family, Corrections, and Workplace as programs are important because a student should have these designated in order to populate their corresponding NRS Outcomes Tables (Table 8-Family, Table 9-Workplace, and Table 10-Correctional Literacy). Correctional students can also be tracked via the Correctional (Yes/No) field in the Demographics folder.

You can add new records or delete records in the Program folder to allow you to track if a student is in multiple programs, since the Primary Program only allows one program, but for federal reporting purposes, it is unnecessary since the Program information for Table 3 is based on the Entry EFL.

The ESL checkbox does not affect Program directly, but can be simply another way to obtain selections of ESL students; however, if you use CASAS assessments that can produce either an ABE or ESL level, then the ESL checkbox is used to determine the appropriate level. If the ESL box is checked, EFLs will be produced as ESL, if that assessment can be used for ESL. If the ESL checkbox is left blank, the EFLs produced will be ABE or ASE.

3. OK, so having said that, how do I make sure my student’s Entry EFL is populating the right EFL? Sometimes I put in an ESL assessment and an ABE assessment, or an assessment such as CASAS Life & Work that can produce either an ESL or an ABE level?

First, always bear in mind that a student can only be reported in ONE subject area for ONE program in any fiscal year, even if within your agency, you are instructing and assessing them in multiple areas. We get it; you don’t only teach them one thing. But from a federal perspective, you can only REPORT one thing, and that is typically the subject area that produces the lowest Educational Functioning Level. If two subject area assessments are entered with different dates, then the first assessment entered will populate the EFL and subject area, so the assessment with the lower EFL should be entered first, or you should use the Select Subject box to select the later assessment if it has the lower EFL (see the Select Subject box instructions below.) If two or more subject areas have the same lowest EFL on the same date, the database breaks that tie by using the lowest Scaled Score or GLE.  There can be occasional exceptions to the lowest EFL, but they should be rare, and documented. An example of a legitimate exception would be a student whose lowest EFL is Reading but they tell you they are only interested in studying Math because that’s the only thing they still need to pass their GED. In that case, you can manually override the database selection of Reading and instead track in Math since that is the area of greatest student need. Be sure to document the reason in the Comments folder so that there is no question of why the exception was made.

To override a subject area in LACES, go to the Assessment folder and double-click the assessment record for the subject area in which you WANT to track the student. In the assessment record, click the Select Subject box. Save the record, and that subject will now be the tracked subject area. You can also check the Select Subject box at the time you are entering the assessment if you know you plan to track the student in that subject area.

Typically, once you populate the Entry Level and Subject Area, that is what the student should remain it for the duration of the fiscal year. An exception to this is if the student shows an Entry Level of Completed Advanced ESL. Students with an Entry Level of Completed Advanced ESL will not be considered NRS reportable/fundable/participant students, because they have already entered as Completed and cannot show growth. So for those students, you should either re-assess them using a different ESL assessment if you initially administered a Best Plus or TABE CLAS-E, or give them an ABE assessment if you feel that is the more appropriate decision. Enter the new assessments, using the Select Subject box to be sure LACES tracks using the new assessment data so that the Entry Level is no longer Completed Advanced ESL.  If you use CASAS assessments that can produce either ESL or ABE levels, the level will be based on whether the ESL checkbox was checked at the time the assessment was entered into LACES. If the ESL checkbox is checked, the level produced will be ESL. If it is Completed Advanced ESL, you can convert to an ASE level by going to the Education folder and unchecking the ESL checkbox. Then, go back into the Assessments folder and force the database to re-read the information based on the updated non-ESL status by opening the assessment record and re-entering the Scaled Score or GLE with the exact same data, but which will now populate an ASE level.

4. But won’t it autoamtically put my ESL student into the Obtain GED goal category if I track them as ABE/ASE instead of Completed Advanced ESL? They already have a diploma from their home country, so I don’t want them in the Obtain GED category.

No, a student’s entry EFL will not place them into an Obtain GED cohort. Only evidence of their having attempted to pass a GED/HiSet/TASC exam or obtain a high school diploma/certification/equivalancy/etc. while enrolled in your program will put them in a GED cohort. Please see question/answer #5 for more information on goals and cohorts.

5. I want to create a Custom View from my student grid that shows their assessment data, but I don’t see those fields in the Custom View options. How can I add them? (Same question asked for adding Comments to a view from the student Comments folder.)

Currently, LACES is only capable of adding data to a custom view if it is a single value, discrete field. Assessment data (and comments) can contain multiple fields, so they cannot be added to a Custom View from the Student tab. However, you can go to the Secondary Navigation Pane and create Custom Views in the All Assessment tab. The All Assessments tab allows the addition of columns for Scaled Score, GLE, Assessed Date, Assessed Level and all kinds of other assessment data.

To access the All Assessments tab, click the button with the double-headed arrow at the very bottom of your Navigation Pane, under the Workshops tab. You will now see the additional tabs, such as All Assessments, All Enrollments, etc.

For the Comments, go to Print>Print Reports from the student grid. There is a report toward the top called Comments by Date Range. Click once on this report to highlight it and then click either Print PDF or Print Excel, then enter the Date Range and click Print to run a report of all comments within that date range. Remember that reports run based on your starting selection, so be sure the selection contains students for whom you wish to see their comment information.

6. What goals do we have to enter? Table 5 is populated from cohorts, not goals, so do we have to enter anything into the goals folder? And how do those cohorts get there if we’re not entering them?

Although Tables 5, 8, 9, and 10 are now populated based on the Cohort folder (not the goals folder) for the Primary Outcomes, you may continue entering those goals into the goals folder if you want to for your own tracking purposes, or you may choose not to.  However, if the state asks for state goals to be tracked, those must still be manually entered into the Goals folder. Additionally, the Secondary Outcomes on Table 11 (an optional table) still generate based on the Goals folder. These outcomes are:

• Achieved Work-Based Project Learning Goal*
• Left Public Assistance
• Achieved Citizenship Skills
• Increased Involvement in Children's Education**
• Increased Involvement in Children's Literacy Activities**
• Voted or Registered To Vote
• Increased Involvement in Community Activities

Finally, the specific program goals for Table 8 (Family Literacy outcomes) are still generated from the Goals folder and should continue to be added manually. However, please note that many of the Family Literacy goals fall under umbrella goals such as Increase Involvement in Child(s) Literacy Activities or Increase Involvement in Child(s) Education. There is no need to enter all of the family literacy goals, as a student meeting any one of the goals under the umbrella goal will result in a Met Outcome for the umbrella goal.

For additional information regarding Table 11 or Table 8, please go to the Online Documentation link on the User News page and click, then click the National Reporting System (NRS) Documentation link on the left. When the documentation page opens, click the top option of “NRS Tables” and in the resulting window, click the specific table documentation available on the far right.

Regarding cohort data versus goals; changes related to the collection and reporting of follow-up measures went into effect on July 1 of the 2012-2013 fiscal year.  While goals are manually entered and can be updated at any time, cohorts are automatically generated by the database when the student exits the program, provided the criteria for cohort inclusion are met.

LACES will automatically assign students to a cohort

1.      IF they meet the cohort criteria described below

2.      AND have met all NRS requirements to be an NRS participant

3.      AND have exited the program

Entered Employment cohort  

·         ANY student whose employment status was unemployed at the time of entry. 

·         For students continuing from the previous fiscal year, LACES will determine if the student’s earliest work history record upon entry in the new fiscal year indicates unemployed.

Retained Employment cohort  

·         ANY student whose employment status was employed at the time of entry. 

·         ANY students who were unemployed at the time of entry but were employed during the first quarter after leaving the program.

·         For students continuing from the previous fiscal year, LACES will determine if the student’s earliest work history record upon entry in the new fiscal year indicates employed, employed-part, or employed-full.

Obtained a GED or Secondary School Diploma cohort   (Students may achieve the outcome before the end of the reporting period for the program year in which they exit, or through the collection period after exiting in the fiscal year)

·         ANY student who took all 5 GED tests as evidenced by the scores being entered in the Assessment folder, or a Date Earned or Date Taken But Not Passed Date being entered in the Diploma folder

OR

·         ANY students who were enrolled in Adult High School at the High Adult Secondary level. 

OR

o   ANY students who were enrolled in Adult High School who enters the program at an entry level of 11^th or 12^th grade regardless of NRS test score.

OR

·         ANY students who were in the assessment phase of the NEDP (National External Diploma Program) as indicated in the Education folder by entered a Date Entered the Assessment Phase of the NEDP

Entered Postsecondary Education or Training cohort   (Students may achieve the outcome anytime from exit through the end of the following fiscal year)

·         ANY student who earned a secondary credential or diploma while enrolled.

OR

·         ANY student who already had a secondary credential/diploma at the time of entry (whether US Based or Non-US Based).

OR

·         ANY students who were enrolled in a class with the Primary Program type of Transition

The NRS tracks the following cohorts on Tables 5 (Mandatory Goal Table)

• Enter Employment

• Retain Employment
• Obtain GED/Obtain Secondary School Diploma
• Place in Post-Secondary Education/Place in Post-Secondary Training

The non-employment related goals (Obtain GED/Secondary School Diploma, Place in Post-Secondary Education/Training) are tracked for students who are NRS participants and leave with those goals set, in the FY for which you are reporting Table 5.

The employment goals are tracked for preceding data ranges that cross fiscal years.

For the 2014-2015 reporting period, the employment goals exit ranges tracked are:

Enter Employment: 10-1-13 to 9-30-14

Retain Employment: 4-1-13 to 3-31-14

So for employment goals, the student must have been an NRS participant in the period in which they exited to show on Table 5.  As an example, if student Michael Morton exits the program on 9-15-14 with an enter employment cohort indicator, he would need to have earned his 12 hours and valid assessment sometime between 7-1-14 to 9-15-14 to populate to Table 5. If Donald Dupont leaves with a retain employment goal on 5-12-13, he must have earned 12 hours and a valid assessment between 7-1-12 to 5-12-13.

Personal goals and secondary outcome measures should still continue to be entered manually into the goals folder.

The NRS tracks the following goals on Table 8 (Optional Table for Goals set by Adults in a Family Literacy Program):

• Complete an Educational Functioning Level
• Obtain Employment
• Retain Employment
• Obtain GED
• Obtain Post-Secondary School Diploma
• Place in Post-Secondary Education
• Place in Post-Secondary Training
• Helped more frequently with school
• Increased contact with children’s teachers
• More involved in children’s school activities
• Reading to children
• Visiting library
• Purchasing books or magazines

The NRS tracks the following goals on Table 9 (Optional Table for Goals set by Adults in a Workforce Literacy Program):

• Complete an Educational Functioning Level
• Obtain Employment
• Retain Employment
• Obtain GED
• Obtain Post-Secondary School Diploma
• Place in Post-Secondary Education
• Place in Post-Secondary Training

The NRS tracks the following goals on Table 10 (Mandatory Table for Goals set by Adults in a Correctional Education Program):

• Complete an Educational Functioning Level
• Obtain Employment
• Retain Employment
• Obtain GED
• Obtain Post-Secondary School Diploma
• Place in Post-Secondary Education
• Place in Post-Secondary Training

The NRS tracks the following goals on Table 11 (Optional Table for Secondary Outcome Measures):

• Achieve work-based project learner goal
• Leave public assistance
• Achieve citizenship skills
• Increased involvement in child’s literacy activities
• Increased involvement in child’s education
• Voted or registered to vote
• Increased involvement in community activities

7. How can I differentiate students who are at a different site? Can I view the Dashboard only for students at Site A versus Site B?

Unfortunately, the Dashboard displays data for ALL current FY students and cannot currently be run on a specific student selection. However, LACES has a field called “Keyword” that allows you to modify the choices to contain customized data, which could include something like a Site A or Site B designation.

Modifying Choice Lists is one of the most powerful ways that you can customize your database.  Many of the fields in your database that have a drop-down menu can be customized by you.  These menus can be customized by adding new choices, hiding choices, editing existing choices, or changing the order in which choices are displayed.  To modify your choice list, you must access the additional tabs on the Navigation Pane.  Click the button with the double-headed arrow at the very bottom of your Navigation Pane, under the Materials tab.

You will now see additional Navigation Pane Options displayed.  Select the “Agency” tab, and select the “Choice Lists” folder from the available folders.

A list of options will display.  These are all of the available drop-down menu items that can be added to, hidden, or edited.  Scroll down the available options to find what you want to edit, whether it is Keyword or Goals or Hours Type.  If there is no modifier to the word, it will usually refer to the Student, such as Keyword.  Other population Keywords would be listed as TutorKeyword, ClassKeyword, etc.  To add an item to the choice list, click the Add New button and complete the applicable fields in the pop-up box, then click the Add button.  You may have to manually increase the size of the window to see the Add button.  The “Display Order” field lets you choose the numerical order in which you want the new choice list item to display.  Smaller numbers will display first.  Although you cannot delete choice list items, you can make them not visible by deselecting the “Visible” box.  Hiding a choice list item will not remove that item if it has already been added to a record; however it will prevent the item from displaying in the drop-down menu in the future.  Editing an existing item acts retroactively, and will change that item if it has already been added to a record.

Although Keyword is there specifically for agency-specific modifications and can be used without concern for how it may affect NRS tables, it is extremely important to remember that modifying some items in your Choice lists can impact whether data populates to your reports correctly.  Certain fields are directly data-mapped to reports and modifying those items or using another choice list item similar to a default item can impact your reports.  We recommend that only one person in your agency modifies choice lists, so that there is a consistent record of changes made, and contact technical support if you are concerned that modifying the list may affect your tables.  Changes made to the Choice List affect everyone in the agency, not just the user who makes the change.

8. Can I push forward an assessment that was done in February to count toward my new fiscal year after July 1?

The state policy allows you to push forward an assessment if it was done between January 1-June 30 of the preceding fiscal year.  To move an assessment forward, simply click on it once to highlight it, then click the “Move Forward” button, and say “Yes” when the pop-up window appears asking if you are sure you want to move it forward. Moving an assessment forward duplicates the moved-forward assessment, so you do not lose the data from last year, but gain a duplicated record in the new year. There is no point to moving forward more assessments in multiple subject areas- just move forward the most current assessment taken in the subject area for which you will be tracking the student in the new fiscal year. Remember that although you may be instructing the student in more than one subject area, the NRS only allows reporting for the lowest EFL or area of greatest student need, so you may only report based on one subject area in a fiscal year.

Keep in mind the student’s attendance, however. If the student has not been regularly attending since the assessment was done, you should re-assess them with a new pre-test in the new fiscal year instead of moving forward the old assessment.

9. Do the hours between assessments carry over if you push an assessment forward from the preceding fiscal year?

Yes. If an assessment is pushed forward from the prior fiscal year, any hours earned after that assessment but before 6-30 are still eligible to be counted as hours between assessments for re-assessment purposes. So, for example, if a TABE assessment was done on April 15 and 25 hours earned between April 16 and June 30, if that assessment is pushed forward, the student would only need 25 additional hours of instruction in the next fiscal year before being eligible for re-assessment. If the student is given a new assessment in the new FY, their hours between assessments will start from 0, because the new assessment has “re-set” the instructional hours since last assessment to 0. The hours between assessments are not counted as instructional hours for the current fiscal year and will not count toward the 12+ hours needed to be an NRS participant; they are only counted as “instructional hours since last assessment” for the purposes of reaching the minimum number of hours between assessments needed to re-assess. You can verify the hours between assessment by applying a custom view that shows Instructional Hours Since Last Assessment or looking at the field in the Key Info folder of the student.

10. Does the Overall Status have anything to do with where a student populated in terms of the columns for Tables 4 or 5?

No. When a student is exited from all of their enrolled classes, pairs, and groups, their Overall Status will automatically go to Left. Changing the Overall Status to Completed Program has no effect on how they populate NRS Tables 4 or 5. If they meet the outcomes for which they populate column B (people in the cohort) on Table 5, then they will show in Column F (people achieving the cohort) on Table 5. On Table 4, a student will only populate Column D (Completed Level) if they did make a level gain in their tracked assessed subject area, as evidenced by a current FY pre- and post-test resulting in an EFL gain in the Assessment folder.

For Table 5, the students will not populate unless they are an NRS participant in the fiscal year for which they are being reported and have been exited. For Table 4, they can be exited or still enrolled, as long as they are an NRS participant.

11. How do you remember what searches you’ve run? If I see the list of students returned as the result of a search, does it tell me what I did to get them?

Currently, no. You have to remember what searches you’ve conducted associated with the results. In an upcoming update to LACES, we will begin attaching search tiles on the grid that show what search criteria resulted in the results.

12. If I do a Staff Search for the students associated with a specific staff member, won’t it bring back all of their students and not just the ones in a particular class?

Yes, a Staff Search will bring back all students assigned to classes for which that staff member is an assigned instructor. You can use the Quick Field or Quick Select Search to remove exited students, or use the Class Search or Class Search-Active Only to search based on just one specific class for that staff member.

13. Since searches and reports are run based on selection, how do I make sure I’m always starting with my current FY students?

From the student grid, go to Find>Current FY. This will always return your current FY students.

14. Does information that I add show up immediately for searches or for the reports?

It depends. Most searches and reports do run off of “raw” data, so it will display immediately for searching or reporting purposes. However, searches such as the FY Data, FY Diagnostics, and NRS FY Table Searches, and all FY NRS Table reports, run off of FY Summary Data, so information used for those searches and reports will not display until a FY Summary has been run.

LiteracyPro automatically creates FY summaries twice a month so that the data gets picked up for the tables, but if you add, modify, or delete data after the FY summaries have been run for the month, you may not see that data populate until the next time summaries are run unless you choose to manually create FY summaries prior to the next update.

1.       The Fiscal Year Summary records are created from the Student or Staff tab and are run on the selection. Since this functionality requires a great deal of processing, we have limited the range of records that can be run. Also, if the selection is generated from the records themselves, you will get an error message letting you know to subset the selection first. You do not need to subset records if you are scheduling them, but to run them yourself, they must be subset first.

2.       To create new FY Summary records, click New on the Toolbar, and then click Create FY Summary.

3.        The selection is limited to 200 records; note the button “Create FY Data Now” is grayed out if the selection is greater, but you can still use the “Schedule” button, or you can limit your selection to 200 persons and run it multiple times for each group of 200.

4.       Select the Fiscal Year on which to run the report. Note: you can currently only run fiscal year records on 14-15.

5.       Because this is an internet application, the browser will time out if the records take too long to process.  Since you may want to create more fiscal year summary records than this, you can schedule FY summaries to be created if you have more than 200 persons in your selection.

6.       If your selection is 200 persons or fewer, click Create FY Data Now.

7.       The button will grey out while the summaries are being created. Because the summary creation is labor-intensive for the database, we recommend that you do not work in LACES until the summaries have been completed.

15.   How can I save the results of a search so I can easily return to it without having to re-run it over and over?

Selection: Allows you to subset, omit, save or load a group of records.  Use the checkmarks to the left of the records you wish to group together, then either subset or omit.  Once you have subset a selection, you can save it to your computer as a document, and then load that group of records at a later date, with current information.

16.   What’s the best way to know when and who to survey?

The state will provide you a list of data-matched student information for the Enter Employment, Retain Employment, and Obtain GED outcome data, so you can go into those student records and update the Cohorts folder. If you need to manually survey for the Enter PS Education/Training cohort or because the student does not have a social security number, there are several ways to do that, depending on what cohort you are searching.

For Employment outcomes for students with no SSN, you can use the Employment Outcome Cohorts for Follow-Up Search in the Student grid. From the Student list, go to Find>Achievements & Goals> Employment Outcome Cohorts for Follow-Up Search. Be sure to select the Search on ALL Students options, as Table 5 data should always be run from ALL students, due to the lag in reporting time for the employment goals. Click the Apply the Above Search Condition button, and then locate the quarter you are currently in (or prior quarters if you are retroactively surveying students) and click the Get Results button. Click the number returned in the Get Results column to be taken to that list of students, where you can apply a view with telephone and other contact information to more easily survey each student. Be aware that you will have to run this search multiple times, as you can only click on one row to get results, and must re-run the search to get back to the results list.

You can also use the All Followup Outcome tab in LACES (accessed by clicking the View Other Tabs button {the one with the little double arrows} at the bottom of the left Navigation Pane to go to the additional tabs) to easily review your goal and cohort information for Table 5.

You cannot edit cohort or goal information from this tab, but it is a good way to review which of your students have met goals or cohort objectives and which are or are not marked as responded to survey.

In the All Followup Outcome tab, you can go to View > Create New Views to create a view that contains SurveyResponded in addition to any other fields you may need, including student contact info.

ALERTS AND MESSAGING

Another great way to know when to do the survey for the Table 5 outcomes is to use the Alerts and Messaging function.

Alerts and Messages is a feature in LACES that allows the user to configure alerts and receive messages in an Inbox with a link to the students who fit the alert criterion.

As part of the Alerts messaging feature, two new tabs have been added:

·         Inbox 

·         Alert Settings 

You can configure the alerts that are relevant to you and schedule when they will be delivered as messages to your LACES inbox or email address.

Configuring and scheduling alerts is optional.  You are not required to utilize the alerts and messages feature.  You can always configure alerts later. You can also disable alerts, or disable notification emails.

 Configuring Alerts

1.  At the Alert Settings tab, double-click an alert record.

2.  Click Edit at the bottom right of the screen.

3.  Add a parameter (alerts that run without parameters will display N/A).  Parameters will either be number of days or number of hours, depending on the alert being configured.

4.  From the drop down Repeats list, select how often you want the alert to run.

5.  Check the box “Send Notification Email” if you’d like to be notified via your email address that an alert has run and has been delivered to your Inbox.  This is optional.

6.  Check the “Enabled” box to receive the alert messages in your Inbox.

7.  Click Save and Return.

TIPS:

• Be sure to check the "Enabled" box as well as the "Send Notification Email" box to receive email notifications.  If only the "Send Notification Email" box is checked, no alerts will be sent to the Inbox or your email address.  You can also skip the Notification Email part if you’ll remember to check the LACES Inbox.
• You must have a current email address associated with your LACES username to receive the email notifications if you check the Send Notification Email box.  To verify or change your email address, go to User->Account and update your email address before checking "Send Notification Email."
• Only one email message will be sent notifying you that new alert messages are in your inbox.  For example, if you scheduled 3 daily alerts, you will receive one email notifying you that all 3 alerts are ready for viewing.  You will not receive a separate email for each alert.

Alerts are run each night on the following schedule:

           

            Daily – every night, including Saturday and Sunday

            Every Weekday (Mon-Fri) – each weekday, excluding Saturday and Sunday

            Every Mon, Wed, Fri

            Every Tues and Thursday

            Weekly – runs on Monday night

            Monthly – runs the first day of the month

            Quarterly – runs the first day of the new quarter (7/1, 10/1, 1/1, 4/1)

            Yearly – runs the first day of the year (1/1)

Changing Alerts

The parameters and repeat schedule can be changed by double-clicking the alert record and editing the Parameter Value field and making a new selection on the drop down list in the Repeats field.  Unchecking the Enabled box will turn off the alert so it will not be run.

TIP:

• Changing the parameters of an alert will not add an additional alert record - there will always be 7 alerts.  The new parameter value will display in the message subject line in the Inbox when the alert is run.  

Alerts

1. Students needing post testing

•  Description: 

1. Current FY students who have met the required number of instructional hours by instrument to be reassessed.
2. Current FY students who have at least one assessment.
3. Current FY students who are active or enrolled, with the most current enrollment date in the current FY year. 

•  Each NRS approved assessment has a recommended number of hours that the student should complete prior to be reassessed.  Unless your state has been customized with different requirements, the number of hours for each instrument are:

Instrument             Minimum recommended hours

TABE Clas-E             50 hours

TABE                         40 hours with the following EFL’s:

                                    Beginning Lit ABE

                                    Beginning ABE

                                    Low Intermediate ABE

                                    High Intermediate ABE

                                 30 hours with the following EFL’s:

                                    Low Adult Secondary

                                    High Adult Secondary

GAIN                         60 hours

BEST Literacy           60 hours

BEST Plus                  60 hours

This alert does not require a parameter, but will run on all current FY.

2. Students not assessed within “x” days of intake

•  Description:  Students whose overall status is prospective or enrolled who have not been assessed in a specified number of days since intake date.

3. Students requiring survey for the entered and retained employment cohorts

• Description:  Students who are ready for enter and retain employment follow-up for the quarter in which the alert is scheduled.

Example:  When the alert is run during the 3rd quarter (1-1 through 3-31) only students who need followed up in the 3rd quarter will be returned.

• The students meeting these criteria can also be found by running the search from All Students->Find->Employment Outcome Cohorts for Follow-up.  The number of students returned in the alert message will match the number of students to contact during the quarter in which the alert was run.    

4. Students requiring survey for the entered postsecondary education and training cohorts

§  Description:  Students who are exited with a postsecondary education and training cohort that has not already been indicated as Met or No with a Responded to Survey indicator

5. Students requiring survey for the Obtained a GED or secondary credential cohorts

§  Description:  Students who are exited with an Obtained a GED or secondary credential cohort that has not already been indicated as Met or No with a Responded to Survey indicator

6. Students with “x” hours prior to assessment eligibility

• Description: Students who are a specified number of hours within the minimum recommended number of hours by instrument for reassessment.

1.  Current FY students who are active or enrolled, with the most current enrollment date in the current FY.
2.  Current FY students with at least one assessment in the current FY.

• Using the chart from #1, students who are within the recommended number of instructional hours by instrument will be returned in the alert message. For example, if the parameter is set for 5 hours, students who have taken a TABE assessment and have completed 35 to 40 instructional hours will be returned in the alert message since they are within 5 hours of reassessment eligibility.

5. Students with no instructional hours after “x” days

•  Description:  Number of days since the student's last instructional hours date.

1. Current FY students who are active or enrolled, with the most current enrollment date in the current FY.
2. When the student is enrolled in more than one class, group, or pair, the last hours date is used. 

Inbox

Alerts are run each morning at 1:00 a.m.  Changes made to the student record after the alert is run will display in the student record, but will not remove the student from the alert.  For example, alert #1 looks for students who are active or enrolled.  Students who are marked left after the alert is run will have left as their overall status.  However, the student was active or enrolled at the time the alert was run.

Alert messages are delivered to the Inbox.  The subject column will display the parameter value entered in the Alert Settings folder, and display a link to the list of students who meet the criteria of the alert.

Alert messages will remain in the Inbox unless you delete them.  This will allow you to see differences in the results based on when the alert was run and allow you to link to the list of students who met alert criteria on an earlier date. 

Additional Information Regarding When to Survey:

Enter Employment: Follow-up must be done in the quarter following exit of the student. The Responded to Survey box should be checked at the time of successful survey, regardless of the outcome. The applicable date range for conducting the follow-up will display in the cohort window when you open it. Table 5 Entered Employment for FY 14/15 is tracking students who exited between October 1, 2013 to September 30, 2014. All data for this outcome is coming only from the cohorts folder and all updates should be made only in the cohorts folder.

Exit Quarter	Follow-Up Quarter
10-1 to 12-31	1-1 to 3-31
1-1 to 3-31	4-1 to 6-30
4-1 to 6-30	7-1 to 9-30
7-1 to 9-30	10-1 to 12-31

Retain Employment: Follow-up must be done in the third quarter following exit of the student. The Responded to Survey box should be checked at the time of successful survey, regardless of the outcome, if you are conducting surveys. The applicable date range for conducting the follow-up will display in the cohort window when you open it.  Table 5 Retained Employment for FY 14/15 is tracking students who exited between 4-1-13 to 3-31-14.  Data for this outcome comes from the cohorts folder and updates should be made in the cohorts folders.

Exit Quarter	Follow-Up Quarter
4-1 to 6-30	1-1 to 3-31
7-1 to 9-30	4-1 to 6-30
10-1 to 12-31	7-1 to 9-30
1-1 to 3-31	10-1 to 12-31

Obtain GED: Follow-up can technically be done at any time up to 12-31 of the calendar year for the fiscal year in which they are placed in the cohort and exit. However, the state typically submits the data around the end of November/beginning of December each year, so you really have the entire fiscal year in which they were placed in the cohort and exited, and then through November following that FY. The applicable date range for conducting the follow-up will display in the cohort window when you open it.  The Responded to Survey box should be checked at population of the cohort.

Enter Post-Secondary Education/Training: Follow-up can be done at any time in the fiscal year in which they are placed in the cohort and exit, and the entire following fiscal year. The applicable date range for conducting the follow-up will display in the cohort window when you open it. The Responded to Survey box should be checked at the time of successful survey, regardless of the outcome, if you are conducting surveys.

All other outcomes: Follow-up can be done at any time in the fiscal year in which they have the outcome and are exited.

Follow-up data is manual. The Survey data must still be manually entered by the user. Survey data should be thought of as the outcome being known, so if you are aware of the outcome (whether the outcome is yes or no), the Responded to Survey box should be checked. Remember that if a cohort is indicated as being Met, but the Responded to Survey box is not checked, the outcome will not populate Table 5 as a met outcome. There is a search for students whose outcome is Yes but the Survey data is not completed, and you can update Survey data as a batch, as explained below.

Included and Responded to Survey Search

The Included and Responded to Survey Search is located in the Find menu in the Student tab. You can choose to search all records or just a current selection (such as Current FY students) and then Apply the Above Search Condition. Then go down the column and click the Get Results button to see students who have met goals or cohorts for the primary outcome measures, but were not included in or responded to survey. Then, after clicking on the number of students returned in the searches, you can use the batch options explained below to update this data as a batch.

Even if a Cohort Record is marked as Met in the Student Record, the objective will not count as Met on Table 5 if Responded to Survey is not checked. Responded to Survey should be checked for all students for whom you know the outcome, however you obtained the data. The only students who should not have Responded to Survey checked are those for whom you do not know the outcome. The students who do not have the Responded to Survey box checked will not be used for calculating Column G, the Met Percentage, so as not to penalize you for not being able to obtain the outcome information for all students; however, please be aware that the Responded to Survey/Data Matched percentage in Column E must be equal to or greater than 50% for OCTAE to consider the data statistically valid. If your Column E percentage statewide is not at or above 50% for all outcomes on Table 5, your data could be considered invalid.

Beginning with 12/13 Reporting, ‘Included in Survey’ is no longer included on Table 5, but we have left it available in LACES for those agencies who want to be able to track that they attempted to contact a student for follow-up even if they were not able to reach the student.

You can update survey data for goals and cohorts as a batch.

Adjust NRS Survey Data-Goals and Cohorts

NRS tables that report measures (Tables 5, 5a, 8, 9, and 10) use calculations to report the percentage of student response rate and percentage of students achieving outcome. These calculations are based on 'Responded to Survey' checkboxes in the student goal and cohort records, therefore it is important that the checkboxes are checked correctly.  The Adjust NRS Survey Data allows users to do batch updates to the survey boxes for students in cohorts or for goals data. 

ONLY students who meet the requirements to be NRS fundable in the 2014-2015 fiscal year or later, and meet the requirements to be included in a cohort (including being left) or who have employment goals being reported on the current FY’s Table 5 will be affected by adjusting the NRS survey data. 

Cohort data will be applicable for students who were NRS fundable in 14/15, met cohort requirements, and exited in 14/15.

To update survey cohort data:

1. Go to the student tab.

2. Move the mouse to New. 

3. Click Adjust NRS Cohort Survey Data.

4. Select the fiscal year from the Left in Fiscal Year dropdown list.  (Students who left prior to the 2012-2013 fiscal year will not be included in a cohort.)

5. Select the choices for 'Included in Survey' and/or 'Responded to Survey.'

6. Check the cohorts to be affected.

7. Apply changes to current selection of students or all students.

IMPORTANT NOTES:  

• Only students who are in a cohort will be affected by adjusting NRS cohort survey data.
• Prior to Adjusting NRS Survey data, the Included and Responded to Survey Search may be used to generate a selection of students who have achieved the cohort outcome but do not have included or responded to survey checked. Additionally, you can go to Find>NRS Table Searches>NRS Table 5 and click in the cells for students in the cohort/having met the cohort and use the results of this search to batch update survey data.

If you have students who you believe should be included on Table 5 as having met one of the objectives and you have verified that responded to survey has been checked, that the met date is correct (for and employment goal), and have update the FY Record, please submit a Tech Support Request and be sure to include the student’s name, the outcome in question, and any other pertinent information.

Additional Table 5 Review

When looking at Table 5, there are multiple columns. An explanation of each column:

Column B is the number of students who exited within the cohort. Think of it as the number of people who left wanting to attain that outcome.

Column C is not applicable to New Mexico because you don’t do random sampling.

Column D are the people you were able to successfully survey (regardless of whether or not they met the outcome) and the people used in a data match. Ideally, you want Column C to be as close as possible to the number in Column B.

Column E shows the percent of people you were able to survey or data match. D/B=% This number must be equal to or greater than 50%, indicating that you were able to data match or survey more than half of the people from Column B, making the results statistically significant

Column F is the number of people who achieved the outcome. A learner cannot be included in F if they were not in Column D, because how can you know they achieved the outcome if you didn’t survey them? That is the main reason it is so important to correctly check the survey box.

Column G is the weighted number of participants achieving the outcome. There are formulas used to “weight” based on survey response.

Finally, Column H is the percentage of people who met the outcome based on number who achieved it and number who responded to survey. Ideally, you want this number to be as high as possible while still being accurate and with a greater than 50% response rate. 

17.