Lab Testing BMM TestLabs Las Vegas, NV June 26 - 28, 2018
NIGA Master Class Seminar Series Gaming Floor and System Audits BMM Testlabs
June 26, 2018
June 27, 2018
June 28, 2018
Sign-in / Registration 8:15 am – 9:00 am
Sign-in 8:30 am – 9:00 am
Sign-in 8:30 am – 9:00 am
• Sign in at BMM Testlabs • Breakfast/snacks
• Arrive at BMM Testlabs
• Arrive at BMM Testlabs
Training Kickoff 9:00 am – 9:15 am
Training Kickoff 9:00 am – 9:15 am
Training Kickoff 9:00 am – 9:45 am
• Review of first day topics • Questions from the attendees
• Review of second day topics • Questions from the attendees
• Introduction by BMMManagement • Introduction of training facilitators • Review of facilities • Overview of the agenda
Overview of an Online Accounting System Audit 9:15 am – 10:30 am • Overview of accounting system components • What is an accounting system audit? • Identifying risk to your accounting system • Data integrity • Developing an accounting system audit plan • When to conduct an accounting system audit
Overview of Class II Systems and Audits 9:15 am – 10:30 am • Review the latest changes to 25 CFR Part 547, and what it means to you. • How Class II systems determine outcome and track accounting • Developing a Class II system audit plan
Overview of a Gaming Floor Audit 9:45 am – 10:30 am
• What is a Gaming Floor Audit? • Identifying risk to your Gaming Floor • Software & Verification • Gaming Devices and associated equipment: o Class III Gaming Devices o Monitoring & Control Systems o Progressive Controllers & Systems o Kiosks o Promotional Systems
Break 10:30 am – 10:45 am Hands on Exercise 10:45 am – 12:00 pm
Break 10:30 am – 10:45 am Hands on Exercise 10:45 am – 12:00 pm
Break 10:30 am – 10:45 am Class II Math, Percentages & Odds 10:45 am – 12:00 pm • Review pay-tables for Class II games
• Software location within gaming machines • Confirmation of software versions • Storage mediums for software programs • Signature verification tools • GAT Verification
• System component verification • Confirmation of software versions • Reports/Logs • Pre-activation testing • Example audit procedures
• Verify Theoretical vs Actual holds • Understand how Bingo outcomes map to entertaining displays.
Lunch 12:00 pm – 1:00 pm Session Review 1:00 pm – 2:00 pm
Lunch 12:00 pm – 1:00 pm Conducting a Gaming Floor Audit 1:00 pm – 2:00 pm
Lunch 12:00 pm – 1:00 pm Overview of a Promotional System Audit 1:00 pm – 2:00 pm • What is a Promotional System audit? • Identifying risk to promotional systems • Risk mitigation
• Review any topics from the first two days • Question and Answer Session
• Communication & Software Risk • Developing a gaming floor audit plan • When to conduct a gaming floor audit • Development of checklists and procedures Break
Break 2:00 pm – 2:15 pm Hands on Exercise 2:15 pm – 3:30 pm
2:00 pm – 2:15 pm Hands on Exercise 2:15 pm – 3:30 pm
• Pre-activation testing • Promotional kiosk verification • Example audit procedures • Reports
• Electronic Gaming Machine configurations • Game Authentication Recap • Machine Emulation/Reel Testing
• Currency Testing • Validation of results
Session Review 3:30 pm – 4:00 pm
Session Review 3:30 pm – 4:00 pm
• Wrap-up of days training • Review overall training • Question and Answer session
• Wrap-up of days training • Question and Answer session • Overview of tomorrow’s topics
6/22/18
Overview of a Gaming Floor Audit
Name of presenter(s) June 2018
Topics of discussion • What’s involved in a Gaming Floor Audit • Identifying risk to your Gaming Floor • Gaming Devices &Associated Equipment • Class II Player Interfaces • Class III Gaming Devices • Monitoring&Control Systems • Progressive Controllers &Systems • Kiosks • Promotional Systems
• Software&Verification
1
6/22/18
What is a gaming floor audit?
• It is a process to check and ensure that the machines and/or tables are setup and operating in amanner expected by the operator, regulator and the public. • When should an audit be conducted? – New games to the floor – Annual audits per MICS & regulation – Disputes/issues
Identifying risk on the gaming floor
• Risk can be foundwithin amodern gaming floor, due tomany reasons – Technology – Cheating or exploits – Maintenance – Etc.
2
6/22/18
What’s involved in a Gaming Floor Audit
• Operational downtime&manpower • Physical inspection of machines • Logical inspection of machines • Report reviewand comparison • Security/Surveillance review
What’s involved in a Gaming Floor Audit
• Operational downtime&manpower – Planning – Resources
3
6/22/18
What’s involved in a Gaming Floor Audit
• Physical Inspections
– Cabinet or table hardware. – Peripherals – Informational – Critical areas – Communication hardware
What’s involved in a Gaming Floor Audit
• Logical Inspections
– Software configurations – Software/firmware versions
– Software Verification – Metering/Accounting – Communication – Artwork / Information
4
6/22/18
What’s involved in a Gaming Floor Audit
• Report Review&Comparison – Accounting information – Significant events
What’s involved in a Gaming Floor Audit
• Security &Surveillance – Seals – Locks
– Camera coverage – *Mind the Gap*
5
6/22/18
What’s involved in a Gaming Floor Audit
• Monitoring &Control Systems – Server Rooms – Environmental controls – Backup and recovery plans – More tomorrow
What’s involved in a Gaming Floor Audit
• Progressives
– System based progressives – Controllers – Signage – Software – Accounting
6
6/22/18
What’s involved in a Gaming Floor Audit
• Kiosks – Promotional, Voucher/ticket, Jackpot – Software – Acceptance and Issuance – Exterior – Communications
Software Verification
• ProgramStorageMedia – Hard disk – CF / CFast – SATADOM • Signature Algorithms – CDCK – Kobetron – SHA1 / HMAC-SHA1 – MD5
7
6/22/18
Software Verification
• SignatureMethods / Tools – Kobetron GI-4000 – Kobetron Verify+ – BMM Signatures – GAT • Other useful equipment
– EPROM reader/burner – Forensic media readers – Windows 10 = Caution!
8
6/22/18
Conducting a Gaming Floor Audit
Name of presenter(s) June 2018
Topics of discussion • Developing a gaming floor audit plan • When to conduct a gaming floor audit • Development of checklists and procedures
1
6/22/18
When to conduct a gaming floor audit?
• Newmachine installs • Machinemoves • Software / machine conversions • Annual machine audit requirements
Developing a Gaming Floor Audit Plan
• Time to conduct the audit • Selection of machines • Selection of personnel • Equipment required • Estimated downtime • Preliminary paperwork • Audit • Reviewreports and determine outcomes
2
6/22/18
Checklists and Procedures
3
6/22/18
Online Accounting System Audit
Nichole Karr, Senior IT Audit Manager June 27, 2018
Topics of discussion • Overview of accounting system and components • What is an accounting system audit • Identifying risk to your accounting system • Data integrity • Developing an accounting system audit plan • When to conduct an accounting system audit
1
6/22/18
Online Accounting Systems Online Accounting Systems are organized sets of computerized accounting methods, procedures, and controls established to gather, record, classify, analyze, summarize, interpret, and present accurate and timely financial data used by all levels of employees and management in a casino environment.
Type of Accounting Systems
In the casino industry, a variety of accounting systems are utilized each day (depending on jurisdictions): • Slots
• Table Games • Cage&Credit • Bingo • Keno • Race&Sports
2
6/22/18
Components of an Accounting System
There are five (5) components that make up an accounting system:
• Source Documents (markers, vouchers, jackpot/fills, table fills/credits)
• Input devices (computers, bar code scanners, slot machines, kiosk)
• Information processors (computers and software programs)
• Information Storage (servers, hard drives)
• Output Devices (Printers, monitors)
Accounting System Audit
Auditing is a systematic and independent examination.
3
6/22/18
Accounting System Audit
The system audits consist of an evaluation of the components which comprise that system, with examination and testing of: • Confirming that systems are appropriately recording and reporting all transactions.
• Validating the correctness of the systems calculations
• Assessing the data integrity of the system
• Verifying that confidential data is not exposed to unauthorized individuals
• Verifying logging of system access and changes to system data
Identifying Risk to Accounting Systems
Accounting systems are fundamental when it comes to casino operations. In the casino industry financial data followed by customer data are what drives every decision make managers, owners, regulators. The information that is captured by the casino’s accounting system is so important that it puts it at a high risk of being lost or stolen.
4
6/22/18
Types of Risks with Accounting Systems There are four general types of risks associated with information technology systems, regardless if they are in development or in use.
• Strategic Risk – Risk of choosing inappropriate technology.
• Operating Risk – the risk of doing the right things in the wrong way. • Financial Risk – the risk of having financial resources lost or stolen. • Information Risk – risk of loss of data integrity, incomplete transactions or hackers.
Data Integrity
Data integrity is the completeness, accuracy and consistency of stored data, without any alteration. Compromised data is little help to companies and major issues can arise from sensitive data loss. For this reason, maintaining data integrity is a core focus of any casino environment.
Data security (act of protecting the data) is one of several measures which can be employed to maintain data integrity.
5
6/22/18
Data Integrity
• Data is expected to be: Attributable, Legible, Contemporaneous, Original, Accurate • Validate computer systems • Implement audit trails • Implement error detection software • Secure records with limited system access • Maintain backup & recovery procedures • Protect the physical & logical security of systems • Properly train users & maintain training records
Accounting System Audit Plan
• Less physical, more logical • Inspect servers, software verification • Reviewreports • Reviewaudit logs
6
6/22/18
When to conduct a system audit?
• Initial installation&configuration • After game installs, conversions, moves • During/After any newcomponents added • Systemupdates/upgrades • Annually
7
6/22/18
Promotional System Audits
Nichole Karr, Senior IT Audit Manager June 2018
Topics of discussion • What is a Promotional System Audit? • Identifying risk to promotional systems • Risk mitigation
1
6/22/18
What is a Promotion System Audit?
• Subset of an Accounting SystemAudit • Customer Service vs. Data &SystemIntegrity • Reconciliation between promotion and accounting system.
Risks to Promotional Systems
• Employee Theft/Collusion • Patron Cheating • Hacking • Configuration / Setup of promotions
2
6/22/18
Risks to Promotional Systems
Tallahassee – A rind of employees and a supervisor at GulfstreamPark Casinomanipulated computer software in the summer of 2007 to steal nearly $290k in slot machine winnings by using cards that allowed themto play for free , according to a state investigation. The casino caught another employee, host S. Dorman, with 17 promotional cards – free play cards usually issued as customer appreciation rewards or marketing tools. Investigators also initially suspected the casino’s vice president of gaming of circumventing internal controls to authorize the fraudulent free play cards .
Risk Mitigation
• User Roles - Segregation of Duties • SystemAccess • Promotional Activity logs/Audit Trails • Daily Audits by Income Control/Accounting • Periodic reviews by Gaming Regulators
3
6/22/18
Class II Systems & Audits
Name of presenter(s) June 2018
Topics of discussion • Latest Changes to 25 CFR Part 547 • How Class II systems determine outcome & track accounting • Developing a Class II audit plan
1
6/22/18
Changes
Major changes from2008 to 2012 to 2018 • No minimum probability requirements – manufacturers and ITL required to disclose game math to TGRA • Grandfather period increased to 10 years from November 2008, and now removed and replaced with Audit requirements • Remote access requirements moved to Part 543 • Removed UL and FCC testing requirements • References to entertaining displays removed
New Grandfathering
• Grandfather systems are nowto be called systems manufactured prior toNovember 2008. • Changes still must maintain or improve compliancewith full Class II requirements • 10%of all previous “grandfather” equipment
must be audited annually. May count towards the entire annual floor audit.
2
6/22/18
How Class II systems work
• Central Determination – Game of Bingo, Pull-tabs, or Lotto • Ball draw performed by a server and provided to each terminal/client • Bingo/Lotto cards either provided by server or generated by each terminal/client • Results often maintained on the server, not on each terminal/client. • Terminal/clients may store accounting, performance and events.
Class II System Audits
FromPart 543: • Should include other external standards such as GAAP, GAAS, etc. • Accounting • Internal Audit • Annual requirements
3
6/22/18
Accounting
• Record gaming activity froman accounting systemto identify and track all revenues, expenses, assets, liabilities, and equity • Record all markers, IOU’s, returned checks, held checks, or other instruments • Record journal entries prepared by the gaming operation and/or any independent accountants
Accounting
• Prepare income statements and balance sheets • Prepare appropriate ledgers to support balance sheets • Reviewsegregated functions, duties, and responsibilities • Reviewminimumbankroll calculations • Reviewpreservation of financial records and supporting documentation
4
6/22/18
Internal Audit
• Bingo
– Supervision – Bingo cards and sales – Draw – Prize payout – Cash & equivalent controls – Technologic aids to the play of Bingo – Operations – Vouchers/Tickets – Revenue audit procedures
Internal Audit
• Pull-tabs
– Supervision – Pull-tab inventory & sales – Winning pull-tabs – Pull-tab operating funds – Statistical records – Revenue audit procedures
5
6/22/18
Internal Audit
• Card games – Supervision
– Exchange or transfers – Playing cards – Shill funds – Reconciliation of card room bank – Posted rules – Progressive pots and pools
Internal Audit
• Information Technology – Supervision
– Class II logical &physical controls – Independence – Physical & logical security – User controls – Installations and/or modifications – Remote access controls – Incident monitoring and reporting – Data backups – Software downloads
6
6/22/18
Internal Audit
• Gaming promotions • Player tracking procedures • Complimentary services or items • Patron deposit accounts • Lines of credit • Drop and count standards • Cage, vault, cash&equivalent procedures • Accounting standards
7
6/22/18
UNDERSTANDING MATH ANALYSIS FOR PAYOUT PERCENTAGES AND ODDS
JUNE LIGHT – DIRECTOR MATHEMATICS, BMM TESTLABS
Overview Why is Class II Math important? How is Class II Math is evaluated?
• Understanding of PAR • What is a PAR Sheet • Primary Uses • Terminology/Components of a PAR Sheet • Volatility • PAR Sheet evaluation
1
6/22/18
Purpose of Class II Math Evaluations
Two Primary Purposes: 1. Verify that all jurisdictional requirements are met (i.e. min/max RTP, Odds) 2. Ensure that the manufacturer’s stated math calculations are correct (i.e. RTP, volatility) WHY?
Minimum Math Requirements
25 CFR § 547.4 (a)Fairness. No Class II gaming systemmay cheat or mislead users. All prizes advertisedmust be available towin during the game. A test laboratorymust calculate and/or verify the mathematical expectations of game play, where applicable, in accordancewith themanufacturer stated submission. The results must be included in the test laboratory’s report to the TGRA. At the request of the TGRA, themanufacturer must also submit themathematical expectations of the game play to the TGRA.
2
6/22/18
Minimum Math Requirements
25 CFR § 547.4 (a)Fairness. No Class II gaming systemmay cheat or mislead users. All prizes advertisedmust be available towin during the game.
Why is Class II Math important?
25 CFR § 547.4 (a)Fairness. No Class II gaming systemmay cheat or mislead users. All prizes advertisedmust be available towin during the game. A test laboratorymust calculate and/or verify the mathematical expectations of game play, where applicable, in accordancewith themanufacturer stated submission. The results must be included in the test laboratory’s report to the TGRA. At the request of the TGRA, themanufacturer must also submit themathematical expectations of the game play to the TGRA.
3
6/22/18
Why is Class II Math important?
25 CFR § 547.4 (a)Fairness. No Class II gaming systemmay cheat or mislead users. All prizes advertisedmust be available towin during the game. A test laboratorymust calculate and/or verify the mathematical expectations of game play, where applicable, in accordancewith themanufacturer stated submission. The results must be included in the test laboratory’s report to the TGRA. At the request of the TGRA, themanufacturer must also submit themathematical expectations of the game play to the TGRA.
Why is Class II Math Important?
25 CFR § 547.16 (c) Odds notification. If the odds of winning any advertised top prize exceeds 100 million to one, the Player Interfacemust display: “Odds of winning the advertised top prize exceeds 100million to one” or equivalent.
4
6/22/18
What does PAR stand for?
Several Different Acronyms • Percentage of Actual Return • Pay Table And Reel Strips • Player Average Return • Probability Accounting Report
What is a PAR sheet? What is a PAR sheet?
• Details howthemath of a bingo game (or similar) is designed • Prize Schedule, Bingo Patterns, andGame Information • Hit rates, progressive info, and top awards • Game volatility and performance characteristics Typically 2 Types: • Detailed PAR sheets - demonstrate howeach mathematical calculationwas derived in the game • Operator/Summary PAR sheets - summarize the key aspects of the product
5
6/22/18
Primary Uses for PAR sheets
Regulatory Uses
• Verifies that the Top Award is compliant • Prize Schedule, Bingo Patterns, and Game Information • Testing and Certification of the product Operational Uses • Analyze slot performance (Theo vs. Actual) • Game performance over time • Setup and configuration of the product Strategic Uses • Segment marketing • (Re)Configuration of the games to maximize revenue
Terminology / Components of a PAR sheet
General Game information – Game theme name, Prize Schedule, Bingo Patterns, and Game Information RTP – Return to player, two main types: theoretical and actual Theoretical RTP - The % of all wagers paid back to the player in a complete game cycle Actual RTP - The % of all wagers actually paid back to players on a particular game, calculated by (money out/money in) Theoretical House Edge – the amount the house will theoretically receive, typically calculated by 100% - Theoretical RTP Odds (expressed as 1 in number) – How many times a particular combination (either prize or award) occurs over the cycle of the game. Ex. If a specified prize can occur 500 times from a game cycle of 10,000 then the odds are 1 in 21 (20+1)
6
6/22/18
Terminology / Components of a PAR sheet
Cycle - The number of all possible unique bingo game outcomes
Probability - The probability that of an outcome occurring (inverse of the odds) Expected Value (EV) - The probability of an event multiplied by the award for that event
Hits – the number of occurrences of a bingo pattern within the game cycle
Hit Rate – The amount of games expected to trigger a particular outcome
Volatility
Game Volatility: Winning/losing behavior over a period of time, varies by how the game math is structured Example: Dice game, three different pay tables, same RTP (not craps) pays for rolling a pair of dice
Paytable1
Paytable2
Paytable3
Outcome Hits
Pay
TotalPay EV
Outcome Hits
Pay
TotalPay EV
Outcome Hits
Pay
TotalPay EV
2 3 4 5 6 7 8 9
2 3 4 5 6 7 8 9
2 3 4 5 6 7 8 9
1 2 3 4 5 6 5 4 3 2 1
1 1 1 1 1 0 1 1 1 1 1
1 2.78% 2 5.56% 3 8.33% 4 11.11% 5 13.89% 0 0.00% 5 13.89% 4 11.11% 3 8.33% 2 5.56% 1 2.78%
1 2 3 4 5 6 5 4 3 2 1
0 0 0 0 0 5 0 0 0 0 0
0 0.00% 0 0.00% 0 0.00% 0 0.00% 0 0.00% 30 83.33% 0 0.00% 0 0.00% 0 0.00% 0 0.00% 0 0.00%
1 2 3 4 5 6 5 4 3 2 1
3 2 1 0 0 0 0 0 1 1
3 8.33% 4 11.11% 3 8.33% 0 0.00% 0 0.00% 0 0.00% 0 0.00% 0 0.00% 3 8.33% 2 5.56% 15 41.67%
10 11 12
10 11 12
10 11 12
15
Total
36 30
RTP
83.33%
Total
36
RTP
83.33%
Total
36 12
RTP
83.33%
Winning
Winning
6
Winning
Combinations
Combinations
Combinations
WinRate
83%
WinRate
17%
WinRate
33%
7
6/22/18
Volatility Index Expected performance of the machine based on the number of games played. Based on a confidence interval. Allows you to compare actual performance to theoretical performance.
PAYTABLE 1 90% Volatility Index
95% Volatility Index
Games
Min % Max % 80.57% 100.23%
Games
Min % Max % 78.72% 102.08% 86.70% 94.09% 89.23% 91.57%
1,000
1,000
10,000 87.29% 93.51% 100,000 89.41% 91.38% 1,000,000 90.09% 90.71% 10,000,000 90.30% 90.50%
10,000 100,000
1,000,000 90.03% 90.77% 10,000,000 90.28% 90.51%
8
6/22/18
PAR Sheet Evaluation - Bingo
Bingo Prize Schedule
PAR Sheet Evaluation - Bingo
9
6/22/18
PAR Sheet Evaluation - Bingo Bingo Prize Schedule
PAR Sheet Evaluation - Bingo
10
6/22/18
PAR Sheet Evaluation - Bingo
PAR Sheet Evaluation - Bingo
11
6/22/18
PAR Sheet Evaluation - Bingo • (c) Odds notification. If the odds of
winning any advertised top prize exceeds 100 million to one, the Player Interface must display: “Odds of winning the advertised top prize exceeds 100 million to one” or equivalent.
Bingo Math • The math behind bingo is important. • How can it be used on the floor?
PAYTABLE 1 90% Volatility Index
95% Volatility Index
Games
Min % Max % 80.57% 100.23% 87.29% 93.51%
Games
Min % Max % 78.72% 102.08% 86.70% 94.09% 89.23% 91.57% 90.03% 90.77%
1,000
1,000
10,000
10,000 100,000
100,000 89.41% 91.38% 1,000,000 90.09% 90.71% 10,000,000 90.30% 90.50%
1,000,000
10,000,000 90.28% 90.51%
12
6/22/18
Final Thoughts
PAR Sheets are a powerful operational tool • Monitor performance • Know what to expect from the machine Strategic use of PAR Sheets may require the assistance of data analytics and experimentation • What type of play am I getting on my games now? • Are my games high volatility or low volatility • What type of customers do I have? • What type of games do my customers prefer to play? • How does floor traffic play into this?
13
Page 1 Page 2-3 Page 4-5 Page 6-7 Page 8 Page 9 Page 10 Page 11 Page 12 Page 13 Page 14 Page 15 Page 16 Page 17 Page 18 Page 19 Page 20 Page 21 Page 22 Page 23 Page 24 Page 25 Page 26 Page 27 Page 28 Page 29 Page 30 Page 31 Page 32 Page 33 Page 34 Page 35 Page 36 Page 37 Page 38 Page 39 Page 40 Page 41 Page 42 Page 43 Page 44 Page 45 Page 46 Page 47 Page 48Made with FlippingBook Annual report